On Mon, Jul 11, 2016 at 05:26:14PM -0400, James Hayden wrote:
Thanks. Added! How often are crawled sites re-indexed?
Will continue adding the other locations now. Looking forward to contributing!
So, you hit a bug in our crawler:
https://github.com/fedora-infra/mirrormanager2/issues/183
I added the corresponding HTTP URLs to your hosts, so that the crawler
is now running. Please add the RSYNC URLs as soon as possible as HTTP
crawling is really ineffective.
You do not need to create a Site for each Host. You can add all Hosts
under one Site. But you can structure it any way you want.
The crawler runs twice a day, but as is could not handle a mirror with
only a HTTPS URL (see issue above) it disabled your mirror after two
days.
Once the mirror is crawled it takes about an hour to re-create the
mirrorlist which is used by the clients. So in few hours you should see
the first clients connecting to your mirror.
Your mirror setup will be interesting as a our crawler will always hit
your LA node, so your CDN and our setup do not make much sense in
combination and crawling all your different URLs will always hit the LA
node. This means that the crawler will crawl multiple times the same
mirror. Let's see how this goes.
Thanks anyway for all your work getting this going for Fedora!
Adrian