4:34 p.m.
Using koji what is the meaning of add-pkg ? I understand I use import
to add a package and tag-pkg or "call tagBuildBypass" to put a tag on
this package. Must I call add-pkg in between?
Thanks,
/Otto
5:18 p.m.
On 12/3/14 2:34 PM, Otto Han wrote:
Using koji what is the meaning of add-pkg ? I understand I use
import
to add a package and tag-pkg or "call tagBuildBypass" to put a tag on
this package. Must I call add-pkg in between?
koji add-pkg adds a package to the whitelist for a tag. Before you can
tag a build into a tag, you need to add the package to the tag whitelist
by calling:
koji add-pkg <tag-name> <package-name> --owner <owner-name>
where <tag-name> is the name of the tag you're trying to tag the build
into, or one of the tags that is inherited by that tag. To see the
current package whitelist for a tag, run:
koji list-pkgs --tag <tag-name>
This will show the full whitelist, including inherited entries.
Thanks,
/Otto
--
buildsys mailing list
buildsys(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys
6:29 p.m.
On Wed, Dec 3, 2014 at 3:18 PM, Mike Bonnet <mikeb(a)redhat.com> wrote:
On 12/3/14 2:34 PM, Otto Han wrote:
>
> Using koji what is the meaning of add-pkg ? I understand I use import
> to add a package and tag-pkg or "call tagBuildBypass" to put a tag on
> this package. Must I call add-pkg in between?
koji add-pkg adds a package to the whitelist for a tag. Before you can tag
a build into a tag, you need to add the package to the tag whitelist by
calling:
koji add-pkg <tag-name> <package-name> --owner <owner-name>
where <tag-name> is the name of the tag you're trying to tag the build into,
or one of the tags that is inherited by that tag. To see the current
package whitelist for a tag, run:
koji list-pkgs --tag <tag-name>
This will show the full whitelist, including inherited entries.
OK. But then what "koji list-untagged" shows? Packages not belonging
to any white list at all or they may be in a white list but not yet
tagged ? If this is true then how can I see packages imported but not
yet included in a white-list.
7:21 p.m.
On 12/3/14 4:29 PM, Otto Han wrote:
On Wed, Dec 3, 2014 at 3:18 PM, Mike Bonnet <mikeb(a)redhat.com>
wrote:
> On 12/3/14 2:34 PM, Otto Han wrote:
>>
>> Using koji what is the meaning of add-pkg ? I understand I use import
>> to add a package and tag-pkg or "call tagBuildBypass" to put a tag on
>> this package. Must I call add-pkg in between?
>
>
> koji add-pkg adds a package to the whitelist for a tag. Before you can tag
> a build into a tag, you need to add the package to the tag whitelist by
> calling:
>
> koji add-pkg <tag-name> <package-name> --owner <owner-name>
>
> where <tag-name> is the name of the tag you're trying to tag the build
into,
> or one of the tags that is inherited by that tag. To see the current
> package whitelist for a tag, run:
>
> koji list-pkgs --tag <tag-name>
>
> This will show the full whitelist, including inherited entries.
>
OK. But then what "koji list-untagged" shows? Packages not belonging
to any white list at all or they may be in a white list but not yet
tagged ? If this is true then how can I see packages imported but not
yet included in a white-list.
koji list-untagged shows *builds* that are not associated with any tag.
This could be because a build was imported and not tagged, or (more
commonly) because a build has been untagged, possibly by the
garbage-collection process (koji-gc).
Example: you want to import findutils-4.5.14-1.fc22 and tag it into the
my-imports tag
$ koji import findutils-4.5.14-1.fc22.src.rpm \
findutils-4.5.14-1.fc22.x86_64.rpm \
findutils-debuginfo-4.5.14-1.fc22.x86_64.rpm
(this creates a findutils-4.5.14-1.fc22 entry in the database,
assuming it doesn't already exist)
$ koji add-pkg my-imports findutils --owner otto
(the argument to --owner would be whatever your koji username is)
$ koji tag-build my-imports findutils-4.5.14-1.fc22
Note that importing rpms isn't common practice. If you're just trying
to get to the point where you can build rpms in Koji, you should be
setting up external repos as outlined here:
https://fedoraproject.org/wiki/Koji/ExternalRepoServerBootstrap
12:57 p.m.
On Wed, Dec 3, 2014 at 5:21 PM, Mike Bonnet <mikeb(a)redhat.com> wrote:
On 12/3/14 4:29 PM, Otto Han wrote:
>
> On Wed, Dec 3, 2014 at 3:18 PM, Mike Bonnet <mikeb(a)redhat.com> wrote:
>>
>> On 12/3/14 2:34 PM, Otto Han wrote:
>>>
>>>
>>> Using koji what is the meaning of add-pkg ? I understand I use import
>>> to add a package and tag-pkg or "call tagBuildBypass" to put a tag
on
>>> this package. Must I call add-pkg in between?
>>
>>
>>
>> koji add-pkg adds a package to the whitelist for a tag. Before you can
>> tag
>> a build into a tag, you need to add the package to the tag whitelist by
>> calling:
>>
>> koji add-pkg <tag-name> <package-name> --owner <owner-name>
>>
>> where <tag-name> is the name of the tag you're trying to tag the build
>> into,
>> or one of the tags that is inherited by that tag. To see the current
>> package whitelist for a tag, run:
>>
>> koji list-pkgs --tag <tag-name>
>>
>> This will show the full whitelist, including inherited entries.
>>
>
> OK. But then what "koji list-untagged" shows? Packages not belonging
> to any white list at all or they may be in a white list but not yet
> tagged ? If this is true then how can I see packages imported but not
> yet included in a white-list.
koji list-untagged shows *builds* that are not associated with any tag.
This could be because a build was imported and not tagged, or (more
commonly) because a build has been untagged, possibly by the
garbage-collection process (koji-gc).
Example: you want to import findutils-4.5.14-1.fc22 and tag it into the
my-imports tag
$ koji import findutils-4.5.14-1.fc22.src.rpm \
findutils-4.5.14-1.fc22.x86_64.rpm \
findutils-debuginfo-4.5.14-1.fc22.x86_64.rpm
(this creates a findutils-4.5.14-1.fc22 entry in the database, assuming
it doesn't already exist)
$ koji add-pkg my-imports findutils --owner otto
(the argument to --owner would be whatever your koji username is)
$ koji tag-build my-imports findutils-4.5.14-1.fc22
Note that importing rpms isn't common practice. If you're just trying to
get to the point where you can build rpms in Koji, you should be setting up
external repos as outlined here:
https://fedoraproject.org/wiki/Koji/ExternalRepoServerBootstrap
--
buildsys mailing list
buildsys(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys
This last sentence is very important for me. We are using Koji as sort
of Package-MgtSystem (package repository). We are building rpm from
srpm in a Jenkins environment. When Jenkins is done and no error was
noted we import all built packages in Koji and tag them.
The source code is maintained in a perforce CMS. When developers reach
a stable phase they generate SRPM from the source code and then run a
Jenkins job to generate the RPM. The result is a temporary local
repository containing all SRPM and RPM of the build. These packages
are then imported in Koji and appropriately tagged:
- koji import *.rpm
- for all untagged: koji call tagBuildBypass tag package
For us importing is a very common practice.
Having said all this what is your advice and what do you think of this
way using Koji ?
Rarely we have a brand new package which does not exist yet in Koji.
Import *.rpm will insert it in Koji but how can I detect I must run
add-pkg for it before tagging?
3:20 p.m.
On 12/04/2014 01:57 PM, Otto Han wrote:
This last sentence is very important for me. We are using Koji as
sort
of Package-MgtSystem (package repository). We are building rpm from
srpm in a Jenkins environment. When Jenkins is done and no error was
noted we import all built packages in Koji and tag them.
The source code is maintained in a perforce CMS. When developers reach
a stable phase they generate SRPM from the source code and then run a
Jenkins job to generate the RPM. The result is a temporary local
repository containing all SRPM and RPM of the build.
I know that folks have wired Jenkins to trigger builds inside of a Koji
instance. I wonder if that would work for you.
These packages
are then imported in Koji and appropriately tagged:
- koji import *.rpm
- for all untagged: koji call tagBuildBypass tag package
For us importing is a very common practice.
If I were doing this sort of mass import all the time. I would first
generate a list of the srpms these rpms were generated from (which
determines the build they will be associated with upon import). E.g.
# rpm --nosignature --nodigest -qp --qf '%{sourcepackage} %{sourcerpm}
%{n}-%{v}-%{r}\n' *.rpm |awk '{if ($1=="1") { print $3 } else {print
gensub(".src.rpm$", "", "", $2)} }'
(The complication above is to deal with the srpm case (just use that
nvr) vs the non-srpm case (use the sourcerpm value).
Anyway, I would save that list and then tag those nvrs after import.
Using list-untagged is not very reliable here, as there are lots of
reasons that a build might become untagged. You might end up tagging
something that you don't mean to.
Having said all this what is your advice and what do you think of
this
way using Koji ?
Well you're not really using most of koji, and you're losing many of the
advantages. With this approach you have very little build metadata in
koji. Garbage collection will be much less intelligent.
Rarely we have a brand new package which does not exist yet in Koji.
Import *.rpm will insert it in Koji but how can I detect I must run
add-pkg for it before tagging?
You could use the koji list-pkgs command to check for it. Or you could
just run koji add-pkg every time. If the package is already present, it
is a no-op (though it will print a warning).
6:11 p.m.
On Fri, Dec 12, 2014 at 1:20 PM, Mike McLean <mikem(a)redhat.com> wrote:
On 12/04/2014 01:57 PM, Otto Han wrote:
> This last sentence is very important for me. We are using Koji as sort
> of Package-MgtSystem (package repository). We are building rpm from
> srpm in a Jenkins environment. When Jenkins is done and no error was
> noted we import all built packages in Koji and tag them.
> The source code is maintained in a perforce CMS. When developers reach
> a stable phase they generate SRPM from the source code and then run a
> Jenkins job to generate the RPM. The result is a temporary local
> repository containing all SRPM and RPM of the build.
I know that folks have wired Jenkins to trigger builds inside of a Koji
instance. I wonder if that would work for you.
> These packages
> are then imported in Koji and appropriately tagged:
> - koji import *.rpm
> - for all untagged: koji call tagBuildBypass tag package
> For us importing is a very common practice.
If I were doing this sort of mass import all the time. I would first
generate a list of the srpms these rpms were generated from (which
determines the build they will be associated with upon import). E.g.
# rpm --nosignature --nodigest -qp --qf '%{sourcepackage} %{sourcerpm}
%{n}-%{v}-%{r}\n' *.rpm |awk '{if ($1=="1") { print $3 } else {print
gensub(".src.rpm$", "", "", $2)} }'
(The complication above is to deal with the srpm case (just use that
nvr) vs the non-srpm case (use the sourcerpm value).
Anyway, I would save that list and then tag those nvrs after import.
Using list-untagged is not very reliable here, as there are lots of
reasons that a build might become untagged. You might end up tagging
something that you don't mean to.
> Having said all this what is your advice and what do you think of this
> way using Koji ?
Well you're not really using most of koji, and you're losing many of the
advantages. With this approach you have very little build metadata in
koji. Garbage collection will be much less intelligent.
> Rarely we have a brand new package which does not exist yet in Koji.
> Import *.rpm will insert it in Koji but how can I detect I must run
> add-pkg for it before tagging?
You could use the koji list-pkgs command to check for it. Or you could
just run koji add-pkg every time. If the package is already present, it
is a no-op (though it will print a warning).
--
Thanks for the help.
If I can abuse of your time... There is a thing I cannot grasp. If I
keep the sources in Koji as SRPM then how do I deal with the project
advancement. We are modifying on a regular base the sources (under
perforce) then we build the SRPM and RPM and import them in Koji. Is
there a Code Management feature in Koji ?
10:23 a.m.
On 12/15/2014 07:11 PM, Otto Han wrote:
On Fri, Dec 12, 2014 at 1:20 PM, Mike McLean <mikem(a)redhat.com>
wrote:
> On 12/04/2014 01:57 PM, Otto Han wrote:
>> This last sentence is very important for me. We are using Koji as sort
>> of Package-MgtSystem (package repository). We are building rpm from
>> srpm in a Jenkins environment. When Jenkins is done and no error was
>> noted we import all built packages in Koji and tag them.
>> The source code is maintained in a perforce CMS. When developers reach
>> a stable phase they generate SRPM from the source code and then run a
>> Jenkins job to generate the RPM. The result is a temporary local
>> repository containing all SRPM and RPM of the build.
>
> I know that folks have wired Jenkins to trigger builds inside of a Koji
> instance. I wonder if that would work for you.
>
>> These packages
>> are then imported in Koji and appropriately tagged:
>> - koji import *.rpm
>> - for all untagged: koji call tagBuildBypass tag package
>> For us importing is a very common practice.
>
> If I were doing this sort of mass import all the time. I would first
> generate a list of the srpms these rpms were generated from (which
> determines the build they will be associated with upon import). E.g.
>
> # rpm --nosignature --nodigest -qp --qf '%{sourcepackage} %{sourcerpm}
> %{n}-%{v}-%{r}\n' *.rpm |awk '{if ($1=="1") { print $3 } else
{print
> gensub(".src.rpm$", "", "", $2)} }'
>
> (The complication above is to deal with the srpm case (just use that
> nvr) vs the non-srpm case (use the sourcerpm value).
>
> Anyway, I would save that list and then tag those nvrs after import.
>
> Using list-untagged is not very reliable here, as there are lots of
> reasons that a build might become untagged. You might end up tagging
> something that you don't mean to.
>
>
>> Having said all this what is your advice and what do you think of this
>> way using Koji ?
>
> Well you're not really using most of koji, and you're losing many of the
> advantages. With this approach you have very little build metadata in
> koji. Garbage collection will be much less intelligent.
>
>> Rarely we have a brand new package which does not exist yet in Koji.
>> Import *.rpm will insert it in Koji but how can I detect I must run
>> add-pkg for it before tagging?
>
> You could use the koji list-pkgs command to check for it. Or you could
> just run koji add-pkg every time. If the package is already present, it
> is a no-op (though it will print a warning).
> --
Thanks for the help.
If I can abuse of your time... There is a thing I cannot grasp. If I
keep the sources in Koji as SRPM then how do I deal with the project
advancement. We are modifying on a regular base the sources (under
perforce) then we build the SRPM and RPM and import them in Koji. Is
there a Code Management feature in Koji ?
Koji is not a source control solution. We assume folks have their own
solution for that. Personally, I like git. Didn't you say you were using
perforce?
11:07 a.m.
On Tue, Dec 16, 2014 at 8:23 AM, Mike McLean <mikem(a)redhat.com> wrote:
On 12/15/2014 07:11 PM, Otto Han wrote:
> On Fri, Dec 12, 2014 at 1:20 PM, Mike McLean <mikem(a)redhat.com> wrote:
>> On 12/04/2014 01:57 PM, Otto Han wrote:
>>> This last sentence is very important for me. We are using Koji as sort
>>> of Package-MgtSystem (package repository). We are building rpm from
>>> srpm in a Jenkins environment. When Jenkins is done and no error was
>>> noted we import all built packages in Koji and tag them.
>>> The source code is maintained in a perforce CMS. When developers reach
>>> a stable phase they generate SRPM from the source code and then run a
>>> Jenkins job to generate the RPM. The result is a temporary local
>>> repository containing all SRPM and RPM of the build.
>>
>> I know that folks have wired Jenkins to trigger builds inside of a Koji
>> instance. I wonder if that would work for you.
>>
>>> These packages
>>> are then imported in Koji and appropriately tagged:
>>> - koji import *.rpm
>>> - for all untagged: koji call tagBuildBypass tag package
>>> For us importing is a very common practice.
>>
>> If I were doing this sort of mass import all the time. I would first
>> generate a list of the srpms these rpms were generated from (which
>> determines the build they will be associated with upon import). E.g.
>>
>> # rpm --nosignature --nodigest -qp --qf '%{sourcepackage} %{sourcerpm}
>> %{n}-%{v}-%{r}\n' *.rpm |awk '{if ($1=="1") { print $3 } else
{print
>> gensub(".src.rpm$", "", "", $2)} }'
>>
>> (The complication above is to deal with the srpm case (just use that
>> nvr) vs the non-srpm case (use the sourcerpm value).
>>
>> Anyway, I would save that list and then tag those nvrs after import.
>>
>> Using list-untagged is not very reliable here, as there are lots of
>> reasons that a build might become untagged. You might end up tagging
>> something that you don't mean to.
>>
>>
>>> Having said all this what is your advice and what do you think of this
>>> way using Koji ?
>>
>> Well you're not really using most of koji, and you're losing many of the
>> advantages. With this approach you have very little build metadata in
>> koji. Garbage collection will be much less intelligent.
>>
>>> Rarely we have a brand new package which does not exist yet in Koji.
>>> Import *.rpm will insert it in Koji but how can I detect I must run
>>> add-pkg for it before tagging?
>>
>> You could use the koji list-pkgs command to check for it. Or you could
>> just run koji add-pkg every time. If the package is already present, it
>> is a no-op (though it will print a warning).
>> --
>
> Thanks for the help.
>
> If I can abuse of your time... There is a thing I cannot grasp. If I
> keep the sources in Koji as SRPM then how do I deal with the project
> advancement. We are modifying on a regular base the sources (under
> perforce) then we build the SRPM and RPM and import them in Koji. Is
> there a Code Management feature in Koji ?
Koji is not a source control solution. We assume folks have their own
solution for that. Personally, I like git. Didn't you say you were using
perforce?
Yes, we use perforce.
I understand importing frequently the SRPM is fine because there is no
way to alter the sources through Koji. Doesn't mean this will somehow
overload the Koji system in time? Is there a way to "purge" the source
RPMs to prevent this situation?
I mean once a build is done there is no reason to keep the sources in
Koji as they are, already in CMS. On the other hand I understand
there is no way to remove / delete a package once in Koji. How can we
deal with this case?
--
buildsys mailing list
buildsys(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/buildsys
11:21 a.m.
On 12/16/2014 12:07 PM, Otto Han wrote:
Yes, we use perforce.
I understand importing frequently the SRPM is fine because there is no
way to alter the sources through Koji. Doesn't mean this will somehow
overload the Koji system in time? Is there a way to "purge" the source
RPMs to prevent this situation?
I mean once a build is done there is no reason to keep the sources in
Koji as they are, already in CMS. On the other hand I understand
there is no way to remove / delete a package once in Koji. How can we
deal with this case?
If you are publishing the srpms, then you probably want to keep them.
They are probably not that large compared to the rpms themselves.
Koji treats builds (the srpm and all the rpms it creates) as a unit. You
can delete old builds as a whole if you need to, or configure the gc to
do so. There is currently no way to delete a single rpm (or srpm) from a
build.
You described a workflow where you build external to koji and imported.
If that is the case, and you don't want the srpms, then perhaps you
should just not import them in the first place. A little extra work will
be required to get imports to work without the srpms. You could either
just add --create-build to the import command to create the build
entries without having the srpm, or you could create them directly with
an api call before importing the rpms.
3437
days inactive
3450
days old
buildsys@lists.fedoraproject.org
9 comments
3 participants
participants (3)
-
Mike Bonnet -
Mike McLean -
Otto Han