Private Koji instance with private SCM: Make sources? Private lookaside?

[Anthony Joseph Messina]

I have have used Git for my RPM sources, patches, specs, etc. for some
time and have just recently installed Koji.  I am able to build from
SRPM (since I'm the admin), but I'd like to connect my Koji instance to
the git scm.

I am currently using the following in kojid.conf:

allowed_scms=messinet.com:/rpms/*:no pkgs.fedoraproject.org:/*:no

When I do this, koji/mock successfully gets the patches and spec file,
but the "Make sources" command doesn't get any of the tar.gz sources.

1) What is the output of the Make sources command (what does it do)?
2) How do I specify a private lookaside cache, similar to what is used
in Fedora?
3) If there is no way to specify a place to get the source tar.gz files,
and I were to create a custom command (rather than Make sources), can I
simply tell it to curl/wget the sources?

Thank you for your help. -A

-- 
Anthony - http://messinet.com - http://messinet.com/~amessina/gallery
8F89 5E72 8DF0 BCF0 10BE 9967 92DC 35DC B001 4A4E

[Steve Traylen]

On Sat, Dec 4, 2010 at 8:08 PM, Anthony Messina <amessina(a)messinet.com&gt; wrote:
>  I have have used Git for my RPM sources, patches, specs, etc. for
> some
>  time and have just recently installed Koji.  I am able to build from
>  SRPM (since I'm the admin), but I'd like to connect my Koji instance to
>  the git scm.
>
>  I am currently using the following in kojid.conf:
>
>  allowed_scms=messinet.com:/rpms/*:no pkgs.fedoraproject.org:/*:no
>
>  When I do this, koji/mock successfully gets the patches and spec file,
>  but the "Make sources" command doesn't get any of the tar.gz sources.
The end result of make sources is that the tar ball must be there, so
the simplest you
can get away with is

sources:
        wget http://yourcache.example.org/software.tar.gz


>  1) What is the output of the Make sources command (what does it do)?
>  2) How do I specify a private lookaside cache, similar to what is used
>  in Fedora?
>  3) If there is no way to specify a place to get the source tar.gz files,
>  and I were to create a custom command (rather than Make sources), can I
>  simply tell it to curl/wget the sources?
>
>  Thank you for your help. -A
>
>  --
>  Anthony - http://messinet.com - http://messinet.com/~amessina/gallery
>  8F89 5E72 8DF0 BCF0 10BE 9967 92DC 35DC B001 4A4E
>
>
>  --
>  buildsys mailing list
>  buildsys(a)lists.fedoraproject.org
>  https://admin.fedoraproject.org/mailman/listinfo/buildsys
>  


-- 
Steve Traylen

[Anthony Joseph Messina]

On 12/04/2010 01:50 PM, Steve Traylen wrote:
>  The end result of make sources is that the tar ball must be there,
> so
>  the simplest you
>  can get away with is
>  
>  sources:
>           
I appreciate the response.  This should get me started.  In the fedora
setup, the 'sources' file for any RPM simply contains an MD5 has and the
name of the source tarball 'software.tar.gz', so is there a Makefile
somewhere that adds the magic for Fedora, like the lookaside cache URL,
etc.?

I have been searching through the code at
https://fedorahosted.org/koji/, but haven't found anything so far.

Also, if I needed to define my own source_cmd, are there variables (like
package name, version, etc.) that I could use so that my uber-simple
command might be something like:

wget http://yourcache.example.org/${package_name}.tar.gz

Thanks.  -A

-- 
Anthony - http://messinet.com - http://messinet.com/~amessina/gallery
8F89 5E72 8DF0 BCF0 10BE 9967 92DC 35DC B001 4A4E

[Steve Traylen]

On Sat, Dec 4, 2010 at 9:34 PM, Anthony Messina <amessina(a)messinet.com&gt; wrote:
>  On 12/04/2010 01:50 PM, Steve Traylen wrote:
> > The end result of make sources is that the tar ball must be there, so
> > the simplest you
> > can get away with is
> >
> > sources:
> >
>
>  I appreciate the response.  This should get me started.  In the fedora
>  setup, the 'sources' file for any RPM simply contains an MD5 has and the
>  name of the source tarball 'software.tar.gz', so is there a Makefile
>  somewhere that adds the magic for Fedora, like the lookaside cache URL,
>  etc.? 
I know how it used to work in the CVS days.
It's just a variable hardcoded in the Makefile that contains
http://somwhere.example.org/lookaside
and then the Makefile parses the "sources" file to get the file name.
The file is curl'd  and then
the md5 sum checked. ... It's about 4 lines longer than the  trivial
solution I gave.

I used to know where the Makefile was in CVS days but someone else
will now reply
as to where the git version is ... or if it now completely different.
   Steve.

>  I have been searching through the code at
>  https://fedorahosted.org/koji/, but haven't found anything so far.
>
>  Also, if I needed to define my own source_cmd, are there variables (like
>  package name, version, etc.) that I could use so that my uber-simple
>  command might be something like:
>
>  wget http://yourcache.example.org/${package_name}.tar.gz
>
>  Thanks.  -A
>
>  --
>  Anthony - http://messinet.com - http://messinet.com/~amessina/gallery
>  8F89 5E72 8DF0 BCF0 10BE 9967 92DC 35DC B001 4A4E
>
>
>  --
>  buildsys mailing list
>  buildsys(a)lists.fedoraproject.org
>  https://admin.fedoraproject.org/mailman/listinfo/buildsys
>  


-- 
Steve Traylen

[Todd Zullinger]

Steve Traylen wrote:
>  I used to know where the Makefile was in CVS days but someone else
>  will now reply as to where the git version is ... or if it now
>  completely different. 
I believe the current code to pull sources from the lookaside case is
in fedpkg now.  The bulk of it is in the sources function in pyfedpkg:

http://git.fedorahosted.org/git/?p=fedora-packager.git;a=blob;f=src/pyfed...

-- 
Todd        OpenPGP -> KeyID: 0xBEAF0CE3 | URL: www.pobox.com/~tmz/pgp
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
No sense being pessimistic, it probably wouldn't work anyway

[Anthony Joseph Messina]

On 12/05/2010 07:39 AM, Todd Zullinger wrote:
>  I believe the current code to pull sources from the lookaside case
> is
>  in fedpkg now.  The bulk of it is in the sources function in pyfedpkg:
>  
>
> http://git.fedorahosted.org/git/?p=fedora-packager.git;a=blob;f=src/pyfed...
Thank you, Todd.  I'll be looking into that.  It seems like using your
own lookaside cache could be a little easier ;)

Well, for now, at least I can build from SRPMs.  Thanks.  -A

-- 
Anthony - http://messinet.com - http://messinet.com/~amessina/gallery
8F89 5E72 8DF0 BCF0 10BE 9967 92DC 35DC B001 4A4E

[Mike McLean]

On 12/04/2010 03:34 PM, Anthony Messina wrote:
>  I have been searching through the code at
>  https://fedorahosted.org/koji/, but haven't found anything so far. 
That is the code for koji, which does not contain any of the code for 
dist-cvs or dist-git.

You can read some about dist-git here:
   http://fedoraproject.org/wiki/Dist_Git_Project

Fedora's dist-git setup makes use of the option in kojid to change the 
'make sources' command to something else via the allowed_scms option. 
They use an entry like this:
   pkgs.fedoraproject.org:/*:false:fedpkg,sources
Which says: allow from pkgs.fedoraproject.org, any path, do not use 
common, run "fedpkg sources" instead of "make sources"

So with Fedora's dist-git, all the magic lies in fedpkg, which is part 
of fedora-packager. You can get that code here:
   https://fedorahosted.org/fedora-packager/

With the old dist-cvs setup, the magic behind 'make sources' was in 
Makefile.common. I'm not sure where that code is archived.

>  Also, if I needed to define my own source_cmd, are there variables
> (like
>  package name, version, etc.) that I could use so that my uber-simple
>  command might be something like:
>
>  wget http://yourcache.example.org/${package_name}.tar.gz 
No, there are no variables.

[Jesse Keating]

On 12/04/2010 11:08 AM, Anthony Messina wrote:
>  1) What is the output of the Make sources command (what does it do)?
>  2) How do I specify a private lookaside cache, similar to what is used
>  in Fedora?
>  3) If there is no way to specify a place to get the source tar.gz files,
>  and I were to create a custom command (rather than Make sources), can I
>  simply tell it to curl/wget the sources? 

Koji configuration allows you to specify which command to issue in order
to get a point where a directory has all the source, spec, and patches
necessary to build an srpm.  It's how we went from doing "make sources"
to "fedpkg sources".  When you are configuring your SCM, you can decide
what "command" you want the system to run in order to get the sources.
It does not have to be make, it does not have to use a "lookaside" of
any sort.  It can be whatever you want, so long as the system is able to
issue a generic command and get the sources pulled down.

-- 
Jesse Keating
Fedora -- Freedom² is a feature!
identi.ca: http://identi.ca/jkeating

[Anthony Joseph Messina]

On 12/06/2010 01:05 PM, Jesse Keating wrote:
>  On 12/04/2010 11:08 AM, Anthony Messina wrote:
> > 1) What is the output of the Make sources command (what does it do)?
> > 2) How do I specify a private lookaside cache, similar to what is used
> > in Fedora?
> > 3) If there is no way to specify a place to get the source tar.gz files,
> > and I were to create a custom command (rather than Make sources), can I
> > simply tell it to curl/wget the sources?
>  
>  
>  Koji configuration allows you to specify which command to issue in order
>  to get a point where a directory has all the source, spec, and patches
>  necessary to build an srpm.  It's how we went from doing "make sources"
>  to "fedpkg sources".  When you are configuring your SCM, you can decide
>  what "command" you want the system to run in order to get the sources.
>  It does not have to be make, it does not have to use a "lookaside" of
>  any sort.  It can be whatever you want, so long as the system is able to
>  issue a generic command and get the sources pulled down.
>   
Thanks to all of you for your help.  After many tries, I was successful.  -A

-- 
Anthony - http://messinet.com - http://messinet.com/~amessina/gallery
8F89 5E72 8DF0 BCF0 10BE 9967 92DC 35DC B001 4A4E