local koji server - packages repository

[Doug Reiland]

I am seeing my builds fail where it knows to pull a previous rpm
locally because it has already built it.
For example, I built kernel.

Now I try to build zlib, and from mock's root.log:

xz-lzma-compat         i686    4.999.9-0.2.beta.20091007git.fc12  build   15 k
DEBUG util.py:256:   zlib                   i686    1.2.3-23.fc12
                build   69 k
DEBUG util.py:256:  Transaction Summary
DEBUG util.py:256:
================================================================================
DEBUG util.py:256:  Install     102 Package(s)
DEBUG util.py:256:  Upgrade       0 Package(s)
DEBUG util.py:256:  Total download size: 78 M
DEBUG util.py:256:
http://localhost/koji/packages/kernel/2.6.31.5/127.fc12/i386/kernel-headers-...:
[Errno 14] HTTP Error 403 :
http://localhost/koji/packages/kernel/2.6.31.5/127.fc12/i386/kernel-headers-...
DEBUG util.py:256:  Trying other mirror.
DEBUG util.py:256:  Error Downloading Packages:
DEBUG util.py:256:    kernel-headers-2.6.31.5-127.fc12.i386: failed to
retrieve kernel-headers-2.6.31.5-127.fc12.i386.rpm from build
DEBUG util.py:256:  error was [Errno 14] HTTP Error 403 :
http://localhost/koji/packages/kernel/2.6.31.5/127.fc12/i386/kernel-headers-...
DEBUG util.py:319:  Child returncode was: 1

I am confused about where this is coming from and if koji serves those
web pages:
My directory is /mnt/koji
> From kojid/kojid.conf: pkgurl=http://localhost/koji/packages
> From /etc/httpd/conf.d/kojiweb.conf:

PythonOption KojiPackagesURL http://localhost/mnt/koji/packages

Any ideas?

[Jeroen van Meeuwen]

On 02/25/2010 02:12 PM, Doug Reiland wrote:
> My directory is /mnt/koji
> > From kojid/kojid.conf: pkgurl=http://localhost/koji/packages
> > From /etc/httpd/conf.d/kojiweb.conf:
>
> PythonOption KojiPackagesURL http://localhost/mnt/koji/packages

I see a subtle mismatch between these two URLs.

-- Jeroen

[Doug Reiland]

I changed the httpd/conf.d/kojiweb.conf to
http://localhost/koji/packages and it didn't help

If I goto http://koji.fedoraproject.org, and click a package, for
example 389-ds, I get to the package page without even logging into.

On my site,

https://localhost/koji/packages/packageinfo?packageID=6

Forbidden

You don't have permission to access /koji/packages/packageinfo on this server.
Apache/2.2.14 (Fedora) Server at localhost Port 443

On Thu, Feb 25, 2010 at 8:31 AM, Jeroen van Meeuwen  wrote:
> On 02/25/2010 02:12 PM, Doug Reiland wrote:
> > My directory is /mnt/koji
> > > From kojid/kojid.conf: pkgurl=http://localhost/koji/packages
> > > From /etc/httpd/conf.d/kojiweb.conf:
> >
> > PythonOption KojiPackagesURL http://localhost/mnt/koji/packages
>
> I see a subtle mismatch between these two URLs.
>
> -- Jeroen

[Dan Horák]

Doug Reiland píše v Čt 25. 02. 2010 v 08:37 -0500:
> I changed the httpd/conf.d/kojiweb.conf to
> http://localhost/koji/packages and it didn't help
>
> If I goto http://koji.fedoraproject.org, and click a package, for
> example 389-ds, I get to the package page without even logging into.
>
> On my site,
>
> https://localhost/koji/packages/packageinfo?packageID=6
>
> Forbidden
>
> You don't have permission to access /koji/packages/packageinfo on this server.
> Apache/2.2.14 (Fedora) Server at localhost Port 443

I have this section in my kojiweb.conf (I use /opt/koji instead of /mnt/koji)

Alias /packages/ "/opt/koji/packages/"

<Directory "/opt/koji/packages/">
    Options None
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>

Dan

[Doug Reiland]

Thanks, Dan. I can now click on a package and go to it's page.
However, my build still fails:

Transaction Summary
DEBUG util.py:256:
================================================================================
DEBUG util.py:256:  Install     102 Package(s)
DEBUG util.py:256:  Upgrade       0 Package(s)
DEBUG util.py:256:  Total download size: 78 M
DEBUG util.py:256:
http://localhost/koji/packages/kernel/2.6.31.5/127.fc12/i386/kernel-headers-...:

If I follow the kernel package down to where its rpms are listed and
try to click on download link ( for headers and example). I get:

Forbidden

You don't have permission to access
/koji/packages/kernel/2.6.31.5/127.fc12/i386/kernel-headers-2.6.31.5-127.fc12.i386.rpm
on this server.

Doug

On Thu, Feb 25, 2010 at 8:49 AM, Dan Horák  wrote:
> Doug Reiland píše v Čt 25. 02. 2010 v 08:37 -0500:
> > I changed the httpd/conf.d/kojiweb.conf to
> > http://localhost/koji/packages and it didn't help
> >
> > If I goto http://koji.fedoraproject.org, and click a package, for
> > example 389-ds, I get to the package page without even logging into.
> >
> > On my site,
> >
> > https://localhost/koji/packages/packageinfo?packageID=6
> >
> > Forbidden
> >
> > You don't have permission to access /koji/packages/packageinfo on this server.
> > Apache/2.2.14 (Fedora) Server at localhost Port 443
>
> I have this section in my kojiweb.conf (I use /opt/koji instead of /mnt/koji)
>
> Alias /packages/ "/opt/koji/packages/"
>
> <Directory "/opt/koji/packages/">
>    Options None
>    AllowOverride None
>    Order allow,deny
>    Allow from all
>
> </Directory>
>
> Dan
>
> --
> buildsys mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/buildsys

[Doug Reiland]

Dan,

Do you mind checking the paths you have for packages in:
/etc/kojid/*.conf
     pkgurl=xx
/etc/http.d/conf.d/kojiweb.conf:

PythonOption KojiPackagesURL http://localhost/koji/packages

On Thu, Feb 25, 2010 at 8:49 AM, Dan Horák  wrote:
> Doug Reiland píše v Čt 25. 02. 2010 v 08:37 -0500:
> > I changed the httpd/conf.d/kojiweb.conf to
> > http://localhost/koji/packages and it didn't help
> >
> > If I goto http://koji.fedoraproject.org, and click a package, for
> > example 389-ds, I get to the package page without even logging into.
> >
> > On my site,
> >
> > https://localhost/koji/packages/packageinfo?packageID=6
> >
> > Forbidden
> >
> > You don't have permission to access /koji/packages/packageinfo on this server.
> > Apache/2.2.14 (Fedora) Server at localhost Port 443
>
> I have this section in my kojiweb.conf (I use /opt/koji instead of /mnt/koji)
>
> Alias /packages/ "/opt/koji/packages/"
>
> <Directory "/opt/koji/packages/">
>    Options None
>    AllowOverride None
>    Order allow,deny
>    Allow from all
>
> </Directory>
>
> Dan
>
> --
> buildsys mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/buildsys

[Dan Horák]

Doug Reiland píše v Čt 25. 02. 2010 v 09:05 -0500:
> Dan,
>
> Do you mind checking the paths you have for packages in:
> /etc/kojid/*.conf
>      pkgurl=xx
> /etc/http.d/conf.d/kojiweb.conf:
>   PythonOption KojiPackagesURL http://localhost/koji/packages

mine is 
PythonOption KojiPackagesURL http://koji.danny.cz/packages

Dan
> On Thu, Feb 25, 2010 at 8:49 AM, Dan Horák  wrote:
> > Doug Reiland píše v Čt 25. 02. 2010 v 08:37 -0500:
> > > I changed the httpd/conf.d/kojiweb.conf to
> > > http://localhost/koji/packages and it didn't help
> > >
> > > If I goto http://koji.fedoraproject.org, and click a package, for
> > > example 389-ds, I get to the package page without even logging into.
> > >
> > > On my site,
> > >
> > > https://localhost/koji/packages/packageinfo?packageID=6
> > >
> > > Forbidden
> > >
> > > You don't have permission to access /koji/packages/packageinfo on this server.
> > > Apache/2.2.14 (Fedora) Server at localhost Port 443
> >
> > I have this section in my kojiweb.conf (I use /opt/koji instead of /mnt/koji)
> >
> > Alias /packages/ "/opt/koji/packages/"
> >
> > <Directory "/opt/koji/packages/">
> >    Options None
> >    AllowOverride None
> >    Order allow,deny
> >    Allow from all
> >
> > </Directory>
> >
> > Dan
> >
> > --
> > buildsys mailing list
> > [e-mail]
> > https://admin.fedoraproject.org/mailman/listinfo/buildsys
>
> --
> buildsys mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/buildsys

[Doug Reiland]

With that change: I go from:

Forbidden

You don't have permission to access
/koji/packages/kernel/2.6.31.5/127.fc12/i386/kernel-headers-2.6.31.5-127.fc12.i386.rpm
on this server.

to:

Not Found

The requested URL
/packages/kernel/2.6.31.5/127.fc12/i386/kernel-headers-2.6.31.5-127.fc12.i386.rpm
was not found on this server.

On Thu, Feb 25, 2010 at 9:15 AM, Dan Horák  wrote:
> Doug Reiland píše v Čt 25. 02. 2010 v 09:05 -0500:
> > Dan,
> >
> > Do you mind checking the paths you have for packages in:
> > /etc/kojid/*.conf
> >      pkgurl=xx
> > /etc/http.d/conf.d/kojiweb.conf:
> >
> > PythonOption KojiPackagesURL http://localhost/koji/packages
>
> mine is
> PythonOption KojiPackagesURL http://koji.danny.cz/packages
>
> Dan
> > On Thu, Feb 25, 2010 at 8:49 AM, Dan Horák  wrote:
> > > Doug Reiland píše v Čt 25. 02. 2010 v 08:37 -0500:
> > > > I changed the httpd/conf.d/kojiweb.conf to
> > > > http://localhost/koji/packages and it didn't help
> > > >
> > > > If I goto http://koji.fedoraproject.org, and click a package, for
> > > > example 389-ds, I get to the package page without even logging into.
> > > >
> > > > On my site,
> > > >
> > > > https://localhost/koji/packages/packageinfo?packageID=6
> > > >
> > > > Forbidden
> > > >
> > > > You don't have permission to access /koji/packages/packageinfo on this server.
> > > > Apache/2.2.14 (Fedora) Server at localhost Port 443
> > >
> > > I have this section in my kojiweb.conf (I use /opt/koji instead of /mnt/koji)
> > >
> > > Alias /packages/ "/opt/koji/packages/"
> > >
> > > <Directory "/opt/koji/packages/">
> > >    Options None
> > >    AllowOverride None
> > >    Order allow,deny
> > >    Allow from all
> > >
> > > </Directory>
> > >
> > > Dan
> > >
> > > --
> > > buildsys mailing list
> > > [e-mail]
> > > https://admin.fedoraproject.org/mailman/listinfo/buildsys
> >
> > --
> > buildsys mailing list
> > [e-mail]
> > https://admin.fedoraproject.org/mailman/listinfo/buildsys
>
> --
> buildsys mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/buildsys

[Doug Reiland]

Well, I am working now.
I package view and download link was fix with added the kojiweb.conf changes.
    - I overlooked a "//opt" from Dan's example which prevented the
download from web page.

The build now works. I am not sure what resolved this, because I think
until a did a regen-repo it may have been using old values.

KojiPackagesURL (in kojiweb.conf) is http://localhost/packages

pkgurl (in kojid/kojid.conf) is http://localhost/packages

Thanks again for everybody's time.

On Thu, Feb 25, 2010 at 9:23 AM, Doug Reiland  wrote:
> With that change: I go from:
>
> Forbidden
>
> You don't have permission to access
> /koji/packages/kernel/2.6.31.5/127.fc12/i386/kernel-headers-2.6.31.5-127.fc12.i386.rpm
> on this server.
>
> to:
>
> Not Found
>
> The requested URL
> /packages/kernel/2.6.31.5/127.fc12/i386/kernel-headers-2.6.31.5-127.fc12.i386.rpm
> was not found on this server.
>
> On Thu, Feb 25, 2010 at 9:15 AM, Dan Horák  wrote:
> > Doug Reiland píše v Čt 25. 02. 2010 v 09:05 -0500:
> > > Dan,
> > >
> > > Do you mind checking the paths you have for packages in:
> > > /etc/kojid/*.conf
> > >      pkgurl=xx
> > > /etc/http.d/conf.d/kojiweb.conf:
> > >
> > > PythonOption KojiPackagesURL http://localhost/koji/packages
> >
> > mine is
> > PythonOption KojiPackagesURL http://koji.danny.cz/packages
> >
> > Dan
> > > On Thu, Feb 25, 2010 at 8:49 AM, Dan Horák  wrote:
> > > > Doug Reiland píše v Čt 25. 02. 2010 v 08:37 -0500:
> > > > > I changed the httpd/conf.d/kojiweb.conf to
> > > > > http://localhost/koji/packages and it didn't help
> > > > >
> > > > > If I goto http://koji.fedoraproject.org, and click a package, for
> > > > > example 389-ds, I get to the package page without even logging into.
> > > > >
> > > > > On my site,
> > > > >
> > > > > https://localhost/koji/packages/packageinfo?packageID=6
> > > > >
> > > > > Forbidden
> > > > >
> > > > > You don't have permission to access /koji/packages/packageinfo on this server.
> > > > > Apache/2.2.14 (Fedora) Server at localhost Port 443
> > > >
> > > > I have this section in my kojiweb.conf (I use /opt/koji instead of /mnt/koji)
> > > >
> > > > Alias /packages/ "/opt/koji/packages/"
> > > >
> > > > <Directory "/opt/koji/packages/">
> > > >    Options None
> > > >    AllowOverride None
> > > >    Order allow,deny
> > > >    Allow from all
> > > >
> > > > </Directory>
> > > >
> > > > Dan
> > > >
> > > > --
> > > > buildsys mailing list
> > > > [e-mail]
> > > > https://admin.fedoraproject.org/mailman/listinfo/buildsys
> > >
> > > --
> > > buildsys mailing list
> > > [e-mail]
> > > https://admin.fedoraproject.org/mailman/listinfo/buildsys
> >
> > --
> > buildsys mailing list
> > [e-mail]
> > https://admin.fedoraproject.org/mailman/listinfo/buildsys

[Doug Reiland]

Is there a way to tell where yum (in a mock environment) pulled a package from?
I will feel better when I validate it is getting the packages (that it
can) locally.
I want to make sure my changes really did fix something and not just
make koji unware of local stuff.

Doug

On Thu, Feb 25, 2010 at 10:07 AM, Doug Reiland  wrote:
> Well, I am working now.
> I package view and download link was fix with added the kojiweb.conf changes.
>    - I overlooked a "//opt" from Dan's example which prevented the
> download from web page.
>
> The build now works. I am not sure what resolved this, because I think
> until a did a regen-repo it may have been using old values.
>
> KojiPackagesURL (in kojiweb.conf) is http://localhost/packages
>
> pkgurl (in kojid/kojid.conf) is http://localhost/packages
>
> Thanks again for everybody's time.
>
> On Thu, Feb 25, 2010 at 9:23 AM, Doug Reiland  wrote:
> > With that change: I go from:
> >
> > Forbidden
> >
> > You don't have permission to access
> > /koji/packages/kernel/2.6.31.5/127.fc12/i386/kernel-headers-2.6.31.5-127.fc12.i386.rpm
> > on this server.
> >
> > to:
> >
> > Not Found
> >
> > The requested URL
> > /packages/kernel/2.6.31.5/127.fc12/i386/kernel-headers-2.6.31.5-127.fc12.i386.rpm
> > was not found on this server.
> >
> > On Thu, Feb 25, 2010 at 9:15 AM, Dan Horák  wrote:
> > > Doug Reiland píše v Čt 25. 02. 2010 v 09:05 -0500:
> > > > Dan,
> > > >
> > > > Do you mind checking the paths you have for packages in:
> > > > /etc/kojid/*.conf
> > > >      pkgurl=xx
> > > > /etc/http.d/conf.d/kojiweb.conf:
> > > >
> > > > PythonOption KojiPackagesURL http://localhost/koji/packages
> > >
> > > mine is
> > > PythonOption KojiPackagesURL http://koji.danny.cz/packages
> > >
> > > Dan
> > > > On Thu, Feb 25, 2010 at 8:49 AM, Dan Horák  wrote:
> > > > > Doug Reiland píše v Čt 25. 02. 2010 v 08:37 -0500:
> > > > > > I changed the httpd/conf.d/kojiweb.conf to
> > > > > > http://localhost/koji/packages and it didn't help
> > > > > >
> > > > > > If I goto http://koji.fedoraproject.org, and click a package, for
> > > > > > example 389-ds, I get to the package page without even logging into.
> > > > > >
> > > > > > On my site,
> > > > > >
> > > > > > https://localhost/koji/packages/packageinfo?packageID=6
> > > > > >
> > > > > > Forbidden
> > > > > >
> > > > > > You don't have permission to access /koji/packages/packageinfo on this server.
> > > > > > Apache/2.2.14 (Fedora) Server at localhost Port 443
> > > > >
> > > > > I have this section in my kojiweb.conf (I use /opt/koji instead of /mnt/koji)
> > > > >
> > > > > Alias /packages/ "/opt/koji/packages/"
> > > > >
> > > > > <Directory "/opt/koji/packages/">
> > > > >    Options None
> > > > >    AllowOverride None
> > > > >    Order allow,deny
> > > > >    Allow from all
> > > > >
> > > > > </Directory>
> > > > >
> > > > > Dan
> > > > >
> > > > > --
> > > > > buildsys mailing list
> > > > > [e-mail]
> > > > > https://admin.fedoraproject.org/mailman/listinfo/buildsys
> > > >
> > > > --
> > > > buildsys mailing list
> > > > [e-mail]
> > > > https://admin.fedoraproject.org/mailman/listinfo/buildsys
> > >
> > > --
> > > buildsys mailing list
> > > [e-mail]
> > > https://admin.fedoraproject.org/mailman/listinfo/buildsys

[Mike McLean]

On 02/25/2010 10:38 AM, Doug Reiland wrote:
> Is there a way to tell where yum (in a mock environment) pulled a package from?
> I will feel better when I validate it is getting the packages (that it
> can) locally.
> I want to make sure my changes really did fix something and not just
> make koji unware of local stuff.

Yum will use the yum config that mock gives it (which comes from the 
mock config). This determines where the packages come from.

I'm a little unclear on your setup. In koji, the mock config will always 
reference a single koji repo. That repo will either be fully internal to 
koji or a merger with one or more external repos.

If you just want to determine which packages in a merged repo are 
external, you can consult the pkgorigins.gz file.

If you want some heavy-handed validation, you might be able to get yum 
to print download sources by cranking up debug output in the yum config 
embedded in the mock config.

[Doug Reiland]

Ok,

Sent from my iPhone

On Feb 25, 2010, at 11:46 AM, Mike McLean  wrote:
> On 02/25/2010 10:38 AM, Doug Reiland wrote:
> > Is there a way to tell where yum (in a mock environment) pulled a  
> > package from?
> > I will feel better when I validate it is getting the packages (that  
> > it
> > can) locally.
> > I want to make sure my changes really did fix something and not just
> > make koji unware of local stuff.
>
> Yum will use the yum config that mock gives it (which comes from the
> mock config). This determines where the packages come from.
>
> I'm a little unclear on your setup. In koji, the mock config will  
> always
> reference a single koji repo. That repo will either be fully  
> internal to
> koji or a merger with one or more external repos.
>
> If you just want to determine which packages in a merged repo are
> external, you can consult the pkgorigins.gz file.
>
> If you want some heavy-handed validation, you might be able to get yum
> to print download sources by cranking up debug output in the yum  
> config
> embedded in the mock config.
> --
> buildsys mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/buildsys

[Doug Reiland]

Ok, I see that now. If this happens again, I hope to be able to dig  
deeper. Looking at pkgorigions.gz under my current repository shows  
location where a package lives

Thanks again

Sent from my iPhone

On Feb 25, 2010, at 11:46 AM, Mike McLean  wrote:
> On 02/25/2010 10:38 AM, Doug Reiland wrote:
> > Is there a way to tell where yum (in a mock environment) pulled a  
> > package from?
> > I will feel better when I validate it is getting the packages (that  
> > it
> > can) locally.
> > I want to make sure my changes really did fix something and not just
> > make koji unware of local stuff.
>
> Yum will use the yum config that mock gives it (which comes from the
> mock config). This determines where the packages come from.
>
> I'm a little unclear on your setup. In koji, the mock config will  
> always
> reference a single koji repo. That repo will either be fully  
> internal to
> koji or a merger with one or more external repos.
>
> If you just want to determine which packages in a merged repo are
> external, you can consult the pkgorigins.gz file.
>
> If you want some heavy-handed validation, you might be able to get yum
> to print download sources by cranking up debug output in the yum  
> config
> embedded in the mock config.
> --
> buildsys mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/buildsys