Yum auto mirroring?

[Arthur Pemberton]

What is the status of mirroring yum? I have two machines (soon to be) with F9.

I would like to speed things up by hosting some form of mirroring on
my Centos5 box.

I am thinking Squid, but not sure if that's the best solution.

Des anyone have success stories? Squid or otherwise? Ideally I could
just set yum only to use a squid port, but yum doesn't seem to
directly support proxies, just indirectly via ENV variables.

I am open to suggestions.

AW: Yum auto mirroring?

[GRAMS, CHRISTIAN]

Why not create a local yum repo? No proxy needed. 
Check this: http://www.howtoforge.com/setting-up-a-local-yum-repository-fedora8

regards
Christian

-----Ursprüngliche Nachricht-----
Von: [e-mail] [mailto:[e-mail]] Im Auftrag von Arthur Pemberton
Gesendet: Dienstag, 1. Juli 2008 07:51
An: For users of Fedora
Betreff: Yum auto mirroring?

What is the status of mirroring yum? I have two machines (soon to be) with F9.

I would like to speed things up by hosting some form of mirroring on
my Centos5 box.

I am thinking Squid, but not sure if that's the best solution.

Des anyone have success stories? Squid or otherwise? Ideally I could
just set yum only to use a squid port, but yum doesn't seem to
directly support proxies, just indirectly via ENV variables.

I am open to suggestions.

[Arthur Pemberton]

On Tue, Jul 1, 2008 at 12:54 AM, GRAMS, CHRISTIAN  wrote:
> Why not create a local yum repo? No proxy needed.
> Check this: http://www.howtoforge.com/setting-up-a-local-yum-repository-fedora8

The problem with that method is the high startup cost and the large
amount of waste

* I have to do a full rsync of all the repos I'll be using
 * there will be a lot of files that I neither want nor need (all of
gnome for example)

I've used this method before, on campus, and even on their fairly
large pipe, the initial rsync of just the main repo (not considering
livna, etc) took quite some time and a lot of hdd space (even with the
proper excludes)

[Matt Domsch]

On Tue, Jul 01, 2008 at 12:50:36AM -0500, Arthur Pemberton wrote:
> What is the status of mirroring yum? I have two machines (soon to be) with F9.
>
> I would like to speed things up by hosting some form of mirroring on
> my Centos5 box.
>
> I am thinking Squid, but not sure if that's the best solution.
>
> Des anyone have success stories? Squid or otherwise? Ideally I could
> just set yum only to use a squid port, but yum doesn't seem to
> directly support proxies, just indirectly via ENV variables.
>
> I am open to suggestions.

https://fedorahosted.org/intelligentmirror/wiki/IntelligentMirror
in in development, and may be of use to you.

[Tim]

On Tue, 2008-07-01 at 00:50 -0500, Arthur Pemberton wrote:
> I would like to speed things up by hosting some form of mirroring on
> my Centos5 box.
>
> I am thinking Squid, but not sure if that's the best solution.
>
> Des anyone have success stories? Squid or otherwise? Ideally I could
> just set yum only to use a squid port, but yum doesn't seem to
> directly support proxies, just indirectly via ENV variables.

Looking at man yum.conf

proxy  url to the proxy server that yum should use.

proxy_username
              username to use for proxy

proxy_password
              password for this proxy

I'm using 7 at the moment, see if Fedora 9 has the same options.  If so,
set each YUM to use your proxy, and always the same mirror (comment out
the mirror list, set pick a specific baseurl URI.

This should work, we used to do the same with Windows to speed up
Windows Update (cache through Squid), it made a huge difference.

[Arthur Pemberton]

On Tue, Jul 1, 2008 at 7:18 AM, Matt Domsch  wrote:
> On Tue, Jul 01, 2008 at 12:50:36AM -0500, Arthur Pemberton wrote:
> > What is the status of mirroring yum? I have two machines (soon to be) with F9.
> >
> > I would like to speed things up by hosting some form of mirroring on
> > my Centos5 box.
> >
> > I am thinking Squid, but not sure if that's the best solution.
> >
> > Des anyone have success stories? Squid or otherwise? Ideally I could
> > just set yum only to use a squid port, but yum doesn't seem to
> > directly support proxies, just indirectly via ENV variables.
> >
> > I am open to suggestions.
>
> https://fedorahosted.org/intelligentmirror/wiki/IntelligentMirror
> in in development, and may be of use to you.

Thanks, I had forgotten the name of that tool.

[Arthur Pemberton]

On Tue, Jul 1, 2008 at 8:18 AM, Tim  wrote:
> On Tue, 2008-07-01 at 00:50 -0500, Arthur Pemberton wrote:
> > I would like to speed things up by hosting some form of mirroring on
> > my Centos5 box.
> >
> > I am thinking Squid, but not sure if that's the best solution.
> >
> > Des anyone have success stories? Squid or otherwise? Ideally I could
> > just set yum only to use a squid port, but yum doesn't seem to
> > directly support proxies, just indirectly via ENV variables.
>
> Looking at man yum.conf
>   proxy  url to the proxy server that yum should use.
>
>   proxy_username
>          username to use for proxy
>
>   proxy_password
>          password for this proxy
>
> I'm using 7 at the moment, see if Fedora 9 has the same options.  If so,
> set each YUM to use your proxy, and always the same mirror (comment out
> the mirror list, set pick a specific baseurl URI.
>
> This should work, we used to do the same with Windows to speed up
> Windows Update (cache through Squid), it made a huge difference.
>
> --
> (This computer runs FC7, my others run FC4, FC5 & FC6, all using Gnome
>  in case that's important to the thread.)
>
> Don't send private replies to my address, the mailbox is ignored.
> I read messages from the public lists.

Great, I knew that was a requested feature for yum , didn't realise
that it had made it in.

Thank you.

[Les Mikesell]

Tim wrote:
> > I would like to speed things up by hosting some form of mirroring on
> > my Centos5 box.
> >
> > I am thinking Squid, but not sure if that's the best solution.
> >
> > Des anyone have success stories? Squid or otherwise? Ideally I could
> > just set yum only to use a squid port, but yum doesn't seem to
> > directly support proxies, just indirectly via ENV variables.
>
> Looking at man yum.conf
>    proxy  url to the proxy server that yum should use.
>
>    proxy_username
>           username to use for proxy
>
>    proxy_password
>           password for this proxy
>
> I'm using 7 at the moment, see if Fedora 9 has the same options.  If so,
> set each YUM to use your proxy, and always the same mirror (comment out
> the mirror list, set pick a specific baseurl URI.
>
> This should work, we used to do the same with Windows to speed up
> Windows Update (cache through Squid), it made a huge difference.

You can always export any variable on the command line like:
http_proxy=http://proxy.domain.com:port_number yum update
for example, to point to a squid configured to cache large files.  The 
problem is that the concept of changing the mirrorlist to a specific URI 
doesn't work well for a set of people who maintain their own machines 
and don't know/care what distribution/version others are using even 
though they are in the same building behind the same proxy cache - and 
it's not that great to have to edit files on every machine to get 
reasonable behavior even if the people do manage to coordinate this.

[Bill Davidsen]

Arthur Pemberton wrote:
> What is the status of mirroring yum? I have two machines (soon to be) with F9.
>
> I would like to speed things up by hosting some form of mirroring on
> my Centos5 box.
>
> I am thinking Squid, but not sure if that's the best solution.
>
> Des anyone have success stories? Squid or otherwise? Ideally I could
> just set yum only to use a squid port, but yum doesn't seem to
> directly support proxies, just indirectly via ENV variables.
>
> I am open to suggestions.

Do an install, set the yum.conf to keep RPMs after use, and upgrade. Now 
link all the rpm files in /var/cache/yum to a single directory and run 
createrepo. You now have a repo of just the stuff you actually found useful.

One the next system, do the install, update from "your" repo, then set 
the "save RPM" flag and update against the usual official repos. Move 
any RPMs you use to the local repo. This will result in pulling anything 
you need only one, and having only things you actually do need. Best of 
all, you can use scripts to do this either automatically or by typing a 
single command, as makes sense for you.

There's a command to prune the local repo to only the latest version of 
things, sorry I don't remember it off the top of my head.

[Tim]

On Tue, 2008-07-01 at 11:17 -0500, Les Mikesell wrote:
> The problem is that the concept of changing the mirrorlist to a
> specific URI  doesn't work well for a set of people who maintain their
> own machines and don't know/care what distribution/version others are
> using even though they are in the same building behind the same proxy
> cache - and it's not that great to have to edit files on every machine
> to get reasonable behavior even if the people do manage to coordinate
> this.

There's ways around that.

A moderately simple thing to do would be make a RPM to set up YUM for
your locale cache.  That gives users a one-step setup.

Alternatively, one could probably get Squid to do something special when
YUM requests the mirror list (re-write the mirror list to suit your
network).