9:19 a.m.
We've installed koji (1.2.5) on F8 and successfully got the
buildroots generating and scratch builds of src.rpms working.
I'm trying to build packages from our git repo. I gave it
the url to our repo:
git://git.fedorahosted.org/git?spacewalk.git#HEAD
But it complains about not having a spec file. Which makes sense
as there is no spec file at the top of the tree. The spec file
I want to build is actually located in the web directory.
http://tinyurl.com/65hs7b
I've tried several versions of the url with no luck. It seems
that koji expects the scm repository house a single package
which is not the case for us.
Has anyone else run into a similar situation? Is there a way
to specify the subdirectory to look in after doing the
git clone?
--
jesus m. rodriguez | jesusr(a)redhat.com
sr. software engineer | irc: zeus
red hat network | 919.754.4413 (w)
rhce # 805008586930012 | 919.623.0080 (c)
+-------------------------------------------+
| "Those who cannot learn from history |
| are doomed to repeat it." |
| -- George Santayana |
+-------------------------------------------+
11:31 a.m.
Jesus M. Rodriguez wrote:
I've tried several versions of the url with no luck. It seems
that koji expects the scm repository house a single package
which is not the case for us.
There are innumerable ways that one might build an srpm from an SCM
checkout. Koji can only support a limited set. Basically, your SCM
layout must support koji, not the other way around.
1:14 p.m.
Mike McLean wrote:
Jesus M. Rodriguez wrote:
> I've tried several versions of the url with no luck. It seems
> that koji expects the scm repository house a single package
> which is not the case for us.
There are innumerable ways that one might build an srpm from an SCM
checkout. Koji can only support a limited set. Basically, your SCM
layout must support koji, not the other way around.
So am I correct in my assumption that koji expects the scm
repository to house a single package?
Is there a way to build from src.rpm without doing scratch
builds? /me couldn't find anything in the docs about this.
Thanks
jesus
--
jesus m. rodriguez | jesusr(a)redhat.com
sr. software engineer | irc: zeus
red hat network | 919.754.4413 (w)
rhce # 805008586930012 | 919.623.0080 (c)
+-------------------------------------------+
| "Those who cannot learn from history |
| are doomed to repeat it." |
| -- George Santayana |
+-------------------------------------------+
1:16 p.m.
On Wed, 2008-07-23 at 14:14 -0400, Jesus M. Rodriguez wrote:
Mike McLean wrote:
> Jesus M. Rodriguez wrote:
>> I've tried several versions of the url with no luck. It seems
>> that koji expects the scm repository house a single package
>> which is not the case for us.
>
> There are innumerable ways that one might build an srpm from an SCM
> checkout. Koji can only support a limited set. Basically, your SCM
> layout must support koji, not the other way around.
So am I correct in my assumption that koji expects the scm
repository to house a single package?
Is there a way to build from src.rpm without doing scratch
builds? /me couldn't find anything in the docs about this.
If you are an admin user in the koji system then you can build from srpm
without doing scratch...but mere mortals aren't allowed that
privilege ;-)
--
Doug Ledford <dledford(a)redhat.com>
GPG KeyID: CFBFF194
http://people.redhat.com/dledford
Infiniband specific RPMs available at
http://people.redhat.com/dledford/Infiniband
8:50 p.m.
Jesus M. Rodriguez wrote:
So am I correct in my assumption that koji expects the scm
repository to house a single package?
Yes. Furthermore, koji assumes that the simple command 'make srpm'
issued from the checkout will create a single source rpm. There is not
enough information passed to koji for it to handle anything otherwise.
In your SCM layout with multiple packages, how do you generate the
different srpms? Is there a real reason that they are all in one module?
Is there a way to build from src.rpm without doing scratch
builds? /me couldn't find anything in the docs about this.
Some of my current work in progress will allow this to be configurable.
Look for this to land in the next few weeks.
At present, as Doug points out, there is an exception for admins.
9:27 p.m.
On Wed, Jul 23, 2008 at 09:50:13PM -0400, Mike McLean wrote:
Jesus M. Rodriguez wrote:
> So am I correct in my assumption that koji expects the scm
> repository to house a single package?
Yes. Furthermore, koji assumes that the simple command 'make srpm'
issued from the checkout will create a single source rpm. There is not
enough information passed to koji for it to handle anything otherwise.
In your SCM layout with multiple packages, how do you generate the
different srpms? Is there a real reason that they are all in one module?
We generate different srpms with the Makefile that exists in each
subdirectory. For example, if you want an srpm for the web module
cd spacewalk.git/web; make srpm
Want one for the java code, then cd spacewalk.git/java.
With SVN it seems completely doable because I could give the
path to repo/spacewalk/java and have koji checkout just that
directory. Seems to be a limitation of GIT in this case
(or we're using it wrong) :)
In theory, if there was a way I could tell koji to pull
from this git repo, and cd into this directory, you'll find
the spec file and Makefile there :)
Is there a real reason for them to be all in one module?
Because this is how we had it in SVN, then switched to git. :)
> Is there a way to build from src.rpm without doing scratch
> builds? /me couldn't find anything in the docs about this.
Some of my current work in progress will allow this to be configurable.
Look for this to land in the next few weeks.
At present, as Doug points out, there is an exception for admins.
For our purposes, we might go the route of the admins.
--
jesus m. rodriguez | jesusr(a)redhat.com
sr. software engineer | irc: zeus
red hat network | 919.754.4413 (w)
rhce # 805008586930012 | 919.623.0080 (c)
+-------------------------------------------+
| "Those who cannot learn from history |
| are doomed to repeat it." |
| -- George Santayana |
+-------------------------------------------+
10:06 p.m.
On Wed, 2008-07-23 at 22:27 -0400, Jesus Rodriguez wrote:
On Wed, Jul 23, 2008 at 09:50:13PM -0400, Mike McLean wrote:
> Jesus M. Rodriguez wrote:
>> So am I correct in my assumption that koji expects the scm
>> repository to house a single package?
>
> Yes. Furthermore, koji assumes that the simple command 'make srpm'
> issued from the checkout will create a single source rpm. There is not
> enough information passed to koji for it to handle anything otherwise.
>
> In your SCM layout with multiple packages, how do you generate the
> different srpms? Is there a real reason that they are all in one module?
We generate different srpms with the Makefile that exists in each
subdirectory. For example, if you want an srpm for the web module
cd spacewalk.git/web; make srpm
Want one for the java code, then cd spacewalk.git/java.
With SVN it seems completely doable because I could give the
path to repo/spacewalk/java and have koji checkout just that
directory. Seems to be a limitation of GIT in this case
(or we're using it wrong) :)
Well, git was intentionally written to be basically a state machine of
the entire project. So, housing more than one project in a single git
repo isn't wrong, but it does tie your various projects together at the
state machine level. This is why you can't checkout just one
subdirectory from git, you have to grab the entire project. I think I
heard mumblings of someone wanting to make subprojects work nicely in
git, but I haven't heard if it ever got off the ground.
Regardless though, koji won't work with what your current SCM layout
without modification to the koji code.
--
Doug Ledford <dledford(a)redhat.com>
GPG KeyID: CFBFF194
http://people.redhat.com/dledford
Infiniband specific RPMs available at
http://people.redhat.com/dledford/Infiniband
10:51 p.m.
On Wed, 2008-07-23 at 23:06 -0400, Doug Ledford wrote:
On Wed, 2008-07-23 at 22:27 -0400, Jesus Rodriguez wrote:
> On Wed, Jul 23, 2008 at 09:50:13PM -0400, Mike McLean wrote:
> > Jesus M. Rodriguez wrote:
> >> So am I correct in my assumption that koji expects the scm
> >> repository to house a single package?
> >
> > Yes. Furthermore, koji assumes that the simple command 'make srpm'
> > issued from the checkout will create a single source rpm. There is not
> > enough information passed to koji for it to handle anything otherwise.
> >
> > In your SCM layout with multiple packages, how do you generate the
> > different srpms? Is there a real reason that they are all in one module?
>
> We generate different srpms with the Makefile that exists in each
> subdirectory. For example, if you want an srpm for the web module
> cd spacewalk.git/web; make srpm
>
> Want one for the java code, then cd spacewalk.git/java.
>
> With SVN it seems completely doable because I could give the
> path to repo/spacewalk/java and have koji checkout just that
> directory. Seems to be a limitation of GIT in this case
> (or we're using it wrong) :)
Well, git was intentionally written to be basically a state machine of
the entire project. So, housing more than one project in a single git
repo isn't wrong, but it does tie your various projects together at the
state machine level. This is why you can't checkout just one
subdirectory from git, you have to grab the entire project. I think I
heard mumblings of someone wanting to make subprojects work nicely in
git, but I haven't heard if it ever got off the ground.
Regardless though, koji won't work with what your current SCM layout
without modification to the koji code.
While that's true, I'm not sure we're definitely handling things
correctly in the git case. The git support in Koji hasn't gotten a lot
of testing, and there are a lot of ways to setup a git repo. However, I
think that multiple projects in the same repo is probably something we
want to support (since we support it in cvs and svn).
The SCM URL you pass to Koji has a format of:
scheme://[user@]host/path/to/repo?path/to/module#revision_or_tag_identifier
With git we combine the /path/to/repo and the path/to/module into a
single URL we pass to "git clone", essentially collapsing them both into
a full module path (this is also what we do with svn). Would it be more
appropriate to treat the /path/to/repo as the location of the git repo
which we would pass to "git clone", and /path/to/module as a
subdirectory in the repo, which we would use as the base directory of
the build, and look there for a .spec file?
7:38 a.m.
On Wed, 2008-07-23 at 23:51 -0400, Mike Bonnet wrote:
On Wed, 2008-07-23 at 23:06 -0400, Doug Ledford wrote:
> On Wed, 2008-07-23 at 22:27 -0400, Jesus Rodriguez wrote:
> > On Wed, Jul 23, 2008 at 09:50:13PM -0400, Mike McLean wrote:
> > > Jesus M. Rodriguez wrote:
> > >> So am I correct in my assumption that koji expects the scm
> > >> repository to house a single package?
> > >
> > > Yes. Furthermore, koji assumes that the simple command 'make srpm'
> > > issued from the checkout will create a single source rpm. There is not
> > > enough information passed to koji for it to handle anything otherwise.
> > >
> > > In your SCM layout with multiple packages, how do you generate the
> > > different srpms? Is there a real reason that they are all in one module?
> >
> > We generate different srpms with the Makefile that exists in each
> > subdirectory. For example, if you want an srpm for the web module
> > cd spacewalk.git/web; make srpm
> >
> > Want one for the java code, then cd spacewalk.git/java.
> >
> > With SVN it seems completely doable because I could give the
> > path to repo/spacewalk/java and have koji checkout just that
> > directory. Seems to be a limitation of GIT in this case
> > (or we're using it wrong) :)
>
> Well, git was intentionally written to be basically a state machine of
> the entire project. So, housing more than one project in a single git
> repo isn't wrong, but it does tie your various projects together at the
> state machine level. This is why you can't checkout just one
> subdirectory from git, you have to grab the entire project. I think I
> heard mumblings of someone wanting to make subprojects work nicely in
> git, but I haven't heard if it ever got off the ground.
>
> Regardless though, koji won't work with what your current SCM layout
> without modification to the koji code.
While that's true, I'm not sure we're definitely handling things
correctly in the git case. The git support in Koji hasn't gotten a lot
of testing, and there are a lot of ways to setup a git repo. However, I
think that multiple projects in the same repo is probably something we
want to support (since we support it in cvs and svn).
Just because these are things that are done in cvs and svn doesn't
necessarily mean we want to do them in git ;-) Amongst other things, if
you put multiple projects into a single repo, then to build *any* of
those projects, you have to clone the *entire* repo. In addition, you
already have to deal with the fact that you are cloning instead of
checking out like you do with cvs. In other words, the overhead of
cloning a huge repo grows much more rapidly for git than for cvs, so the
pain of a multi-project repo is going to be felt much more acutely.
The SCM URL you pass to Koji has a format of:
scheme://[user@]host/path/to/repo?path/to/module#revision_or_tag_identifier
With git we combine the /path/to/repo and the path/to/module into a
single URL we pass to "git clone", essentially collapsing them both into
a full module path (this is also what we do with svn). Would it be more
appropriate to treat the /path/to/repo as the location of the git repo
which we would pass to "git clone", and /path/to/module as a
subdirectory in the repo, which we would use as the base directory of
the build, and look there for a .spec file?
I could certainly see that as being a valid interpretation of the url
string.
--
Doug Ledford <dledford(a)redhat.com>
GPG KeyID: CFBFF194
http://people.redhat.com/dledford
Infiniband specific RPMs available at
http://people.redhat.com/dledford/Infiniband
8:32 a.m.
On Wed, Jul 23, 2008 at 11:51:36PM -0400, Mike Bonnet wrote:
On Wed, 2008-07-23 at 23:06 -0400, Doug Ledford wrote:
> On Wed, 2008-07-23 at 22:27 -0400, Jesus Rodriguez wrote:
> > On Wed, Jul 23, 2008 at 09:50:13PM -0400, Mike McLean wrote:
> > > Jesus M. Rodriguez wrote:
> > >> So am I correct in my assumption that koji expects the scm
> > >> repository to house a single package?
> > >
> > > Yes. Furthermore, koji assumes that the simple command 'make srpm'
> > > issued from the checkout will create a single source rpm. There is not
> > > enough information passed to koji for it to handle anything otherwise.
> > >
> > > In your SCM layout with multiple packages, how do you generate the
> > > different srpms? Is there a real reason that they are all in one module?
> >
> > We generate different srpms with the Makefile that exists in each
> > subdirectory. For example, if you want an srpm for the web module
> > cd spacewalk.git/web; make srpm
> >
> > Want one for the java code, then cd spacewalk.git/java.
> >
> > With SVN it seems completely doable because I could give the
> > path to repo/spacewalk/java and have koji checkout just that
> > directory. Seems to be a limitation of GIT in this case
> > (or we're using it wrong) :)
>
> Well, git was intentionally written to be basically a state machine of
> the entire project. So, housing more than one project in a single git
> repo isn't wrong, but it does tie your various projects together at the
> state machine level. This is why you can't checkout just one
> subdirectory from git, you have to grab the entire project. I think I
> heard mumblings of someone wanting to make subprojects work nicely in
> git, but I haven't heard if it ever got off the ground.
>
> Regardless though, koji won't work with what your current SCM layout
> without modification to the koji code.
While that's true, I'm not sure we're definitely handling things
correctly in the git case. The git support in Koji hasn't gotten a lot
of testing, and there are a lot of ways to setup a git repo. However, I
think that multiple projects in the same repo is probably something we
want to support (since we support it in cvs and svn).
The SCM URL you pass to Koji has a format of:
scheme://[user@]host/path/to/repo?path/to/module#revision_or_tag_identifier
With git we combine the /path/to/repo and the path/to/module into a
single URL we pass to "git clone", essentially collapsing them both into
a full module path (this is also what we do with svn). Would it be more
appropriate to treat the /path/to/repo as the location of the git repo
which we would pass to "git clone", and /path/to/module as a
subdirectory in the repo, which we would use as the base directory of
the build, and look there for a .spec file?
Yep, that would actually work in our case. Because then I could do the
following:
git://git.fedorahosted.org/git/spacewalk.git?web#HEAD
web would contain the .spec file.
--
jesus m. rodriguez | jesusr(a)redhat.com
sr. software engineer | irc: zeus
red hat network | 919.754.4413 (w)
rhce # 805008586930012 | 919.623.0080 (c)
+-------------------------------------------+
| "Those who cannot learn from history |
| are doomed to repeat it." |
| -- George Santayana |
+-------------------------------------------+
3:08 p.m.
On Thu, 2008-07-24 at 09:32 -0400, Jesus Rodriguez wrote:
On Wed, Jul 23, 2008 at 11:51:36PM -0400, Mike Bonnet wrote:
> On Wed, 2008-07-23 at 23:06 -0400, Doug Ledford wrote:
> > On Wed, 2008-07-23 at 22:27 -0400, Jesus Rodriguez wrote:
> > > On Wed, Jul 23, 2008 at 09:50:13PM -0400, Mike McLean wrote:
> > > > Jesus M. Rodriguez wrote:
> > > >> So am I correct in my assumption that koji expects the scm
> > > >> repository to house a single package?
> > > >
> > > > Yes. Furthermore, koji assumes that the simple command 'make
srpm'
> > > > issued from the checkout will create a single source rpm. There is
not
> > > > enough information passed to koji for it to handle anything
otherwise.
> > > >
> > > > In your SCM layout with multiple packages, how do you generate the
> > > > different srpms? Is there a real reason that they are all in one
module?
> > >
> > > We generate different srpms with the Makefile that exists in each
> > > subdirectory. For example, if you want an srpm for the web module
> > > cd spacewalk.git/web; make srpm
> > >
> > > Want one for the java code, then cd spacewalk.git/java.
> > >
> > > With SVN it seems completely doable because I could give the
> > > path to repo/spacewalk/java and have koji checkout just that
> > > directory. Seems to be a limitation of GIT in this case
> > > (or we're using it wrong) :)
> >
> > Well, git was intentionally written to be basically a state machine of
> > the entire project. So, housing more than one project in a single git
> > repo isn't wrong, but it does tie your various projects together at the
> > state machine level. This is why you can't checkout just one
> > subdirectory from git, you have to grab the entire project. I think I
> > heard mumblings of someone wanting to make subprojects work nicely in
> > git, but I haven't heard if it ever got off the ground.
> >
> > Regardless though, koji won't work with what your current SCM layout
> > without modification to the koji code.
>
> While that's true, I'm not sure we're definitely handling things
> correctly in the git case. The git support in Koji hasn't gotten a lot
> of testing, and there are a lot of ways to setup a git repo. However, I
> think that multiple projects in the same repo is probably something we
> want to support (since we support it in cvs and svn).
>
> The SCM URL you pass to Koji has a format of:
>
> scheme://[user@]host/path/to/repo?path/to/module#revision_or_tag_identifier
>
> With git we combine the /path/to/repo and the path/to/module into a
> single URL we pass to "git clone", essentially collapsing them both into
> a full module path (this is also what we do with svn). Would it be more
> appropriate to treat the /path/to/repo as the location of the git repo
> which we would pass to "git clone", and /path/to/module as a
> subdirectory in the repo, which we would use as the base directory of
> the build, and look there for a .spec file?
Yep, that would actually work in our case. Because then I could do the
following:
git://git.fedorahosted.org/git/spacewalk.git?web#HEAD
web would contain the .spec file.
I've just committed a change to Koji git to support this style of repo
setup. The module path (everything between the ? and #) is now
considered a subdirectory of the repo, and Koji will expect to find the
relevant Makefile and specfile there. Building from the previous repo
layout (where the top-level directory contains the Makefile and
specfile) can be achieved by leaving empty or omitting the module path,
like so:
git://git.fedorahosted.org/git/koji?#a9a82be4
git://git.fedorahosted.org/git/koji#a9a82be4
The two examples above are functionally identical. I've tested these
changes against both the Spacewalk and Koji git repos, and they seem to
be working as expected. Let me know if you run into problems.
2:29 a.m.
"Jesus M. Rodriguez" <jesusr(a)redhat.com> writes:
git://git.fedorahosted.org/git?spacewalk.git#HEAD
But it complains about not having a spec file. Which makes sense
as there is no spec file at the top of the tree.
We solved a similar issue (building multiple branches from same sources)
by passing some information to make; e.g. see
http://ensc.de/fedora/0007-pass-some-information-to-the-make-srpm-task.patch
'make' extracts then the branch-id out of the scm revision and set
e.g. correct %dist.
Should be easy to adapt for your case.
Enrico
5734
days inactive
5754
days old
buildsys@lists.fedoraproject.org
11 comments
5 participants
participants (5)
-
Doug Ledford -
Enrico Scholz -
jesus rodriguez -
Mike Bonnet -
Mike McLean