Do you have any sort of rate limiter in place? The MM crawler is able to do a single HTTP
HEAD request against your server every 5 seconds, and it has a lot of HTTP HEAD requests
to do across all those directories. It may simply be timing out.
connect: (mirror.squ.edu.om, 80)
send: u'HEAD /fedora/linux/development/16/x86_64/os/.discinfo HTTP/1.1\r\nHost:
mirror.squ.edu.om\r\nAccept-Encoding: identity\r\nConnection: Keep-Alive\r\nPragma:
no-cache\r\nUser-Agent: mirrormanager-crawler/0.1
(+http://fedorahosted.org/mirrormanager)\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Thu, 03 Nov 2011 16:27:02 GMT
header: Server: Apache/2.2.15 (Red Hat)
header: Last-Modified: Wed, 02 Nov 2011 11:52:04 GMT
header: ETag: "baa031f-23-4b0bf19be8100"
header: Accept-Ranges: bytes
header: Content-Length: 35
header: Connection: close
header: Content-Type: text/plain; charset=UTF-8
connect: (mirror.squ.edu.om, 80)
send: u'HEAD /fedora/linux/development/16/x86_64/os/.treeinfo HTTP/1.1\r\nHost:
mirror.squ.edu.om\r\nAccept-Encoding: identity\r\nConnection: Keep-Alive\r\nPragma:
no-cache\r\nUser-Agent: mirrormanager-crawler/0.1
(+http://fedorahosted.org/mirrormanager)\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Thu, 03 Nov 2011 16:27:07 GMT
header: Server: Apache/2.2.15 (Red Hat)
header: Last-Modified: Wed, 02 Nov 2011 11:54:52 GMT
header: ETag: "baa0320-36d-4b0bf23c1fb00"
header: Accept-Ranges: bytes
header: Content-Length: 877
header: Connection: close
header: Content-Type: text/plain; charset=UTF-8
connect: (mirror.squ.edu.om, 80)
send: u'HEAD /fedora/linux/development/16/x86_64/os/RPM-GPG-KEY-fedora-ppc64
HTTP/1.1\r\nHost: mirror.squ.edu.om\r\nAccept-Encoding: identity\r\nConnection:
Keep-Alive\r\nPragma: no-cache\r\nUser-Agent: mirrormanager-crawler/0.1
(+http://fedorahosted.org/mirrormanager)\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Thu, 03 Nov 2011 16:27:11 GMT
header: Server: Apache/2.2.15 (Red Hat)
header: Last-Modified: Tue, 25 Oct 2011 04:50:12 GMT
header: ETag: "baa460d-eeb-4b018464d0900"
header: Accept-Ranges: bytes
header: Content-Length: 3819
header: Connection: close
header: Content-Type: text/plain; charset=UTF-8
--
Matt Domsch
Technology Strategist
Dell | Office of the CTO
-----Original Message-----
From: SQU Mirror Server [mailto:mirror@squ.edu.om]
Sent: Thursday, November 03, 2011 11:10 AM
To: Domsch, Matt; mirror-admin(a)fedoraproject.org
Subject: RE: Request to add our mirror to your database
I put a network block for our local users in my case Omanis. However, I want every one
benefit from our mirror server with giving high priority to local users. Is this possible
?
Our corn tab for fedora sync:
0 4,12,19,23 * * * /usr/bin/rsync -vaH --numeric-ids --delete --delete-after
--delay-updates
rsync://download.fedora.redhat.com/fedora-enchilada /open_source1/fedora
Is there any wrong in it? However, here is the recent log Starting crawl
2011-11-03T07:00:23.792939 Ending crawl 2011-11-03T07:09:48.638290
Kind regards,
Fahad Al-Saidi
__________________________________
Mirror Admin Team
Center for Information Systems,
Sultan Qaboos University,
P.O. Box 40, Postal Code 123,
Sultanate of Oman.
CIS Telephone number: (+968)24141500
CIS Fax Number: (+968)24413320
________________________________________
From: Matt_Domsch(a)Dell.com [Matt_Domsch(a)Dell.com]
Sent: Thursday, November 03, 2011 8:30 AM
To: SQU Mirror Server; mirror-admin(a)fedoraproject.org
Subject: RE: Request to add our mirror to your database
You do not have any network blocks listed in MM, so it's not going to automatically
route your local users to your mirror.
https://admin.fedoraproject.org/mirrormanager/host/1406
Furthermore, the MM crawler sees you have some directories up-to-date, but not all. This
is in the recent crawler logs. Lines that look like directories are those that the crawler
thinks are up-to-date. It's not obvious from these logs why the crawler doesn't
think most of your releases/ tree is up-to-date, but it doesn't.
Starting crawl 2011-11-02T07:00:23.091279
http://mirror.squ.edu.om/fedora/linux/development/16/i386/debug
http://mirror.squ.edu.om/fedora/linux/development/16/i386/debug/repodata
http://mirror.squ.edu.om/fedora/linux/development/16/i386/os
http://mirror.squ.edu.om/fedora/linux/development/16/i386/os/Packages
http://mirror.squ.edu.om/fedora/linux/development/16/i386/os/drpms
http://mirror.squ.edu.om/fedora/linux/development/16/i386/os/images
http://mirror.squ.edu.om/fedora/linux/development/16/i386/os/images/pxeboot
http://mirror.squ.edu.om/fedora/linux/development/16/i386/os/isolinux
http://mirror.squ.edu.om/fedora/linux/development/16/i386/os/repodata
http://mirror.squ.edu.om/fedora/linux/development/16/source/SRPMS
http://mirror.squ.edu.om/fedora/linux/development/16/source/SRPMS/repodata
http://mirror.squ.edu.om/fedora/linux/development/16/x86_64/debug
http://mirror.squ.edu.om/fedora/linux/development/16/x86_64/debug/repodata
http://mirror.squ.edu.om/fedora/linux/development/16/x86_64/os
http://mirror.squ.edu.om/fedora/linux/development/16/x86_64/os/EFI/BOOT
http://mirror.squ.edu.om/fedora/linux/development/16/x86_64/os/Packages
http://mirror.squ.edu.om/fedora/linux/development/16/x86_64/os/drpms
http://mirror.squ.edu.om/fedora/linux/development/16/x86_64/os/images
http://mirror.squ.edu.om/fedora/linux/development/16/x86_64/os/images/pxe...
http://mirror.squ.edu.om/fedora/linux/development/16/x86_64/os/isolinux
http://mirror.squ.edu.om/fedora/linux/development/16/x86_64/os/repodata
http://mirror.squ.edu.om/fedora/linux/development/rawhide/i386/debug
http://mirror.squ.edu.om/fedora/linux/development/rawhide/i386/debug/repo...
http://mirror.squ.edu.om/fedora/linux/development/rawhide/i386/os/Packages
http://mirror.squ.edu.om/fedora/linux/development/rawhide/i386/os/drpms
http://mirror.squ.edu.om/fedora/linux/development/rawhide/i386/os/repodata
http://mirror.squ.edu.om/fedora/linux/development/rawhide/source/SRPMS
http://mirror.squ.edu.om/fedora/linux/development/rawhide/source/SRPMS/re...
http://mirror.squ.edu.om/fedora/linux/development/rawhide/x86_64/debug
http://mirror.squ.edu.om/fedora/linux/development/rawhide/x86_64/debug/re...
http://mirror.squ.edu.om/fedora/linux/development/rawhide/x86_64/os/Packages
http://mirror.squ.edu.om/fedora/linux/development/rawhide/x86_64/os/drpms
http://mirror.squ.edu.om/fedora/linux/development/rawhide/x86_64/os/repodata
http://mirror.squ.edu.om/fedora/linux/releases/10
http://mirror.squ.edu.om/fedora/linux/releases/11
http://mirror.squ.edu.om/fedora/linux/releases/12
http://mirror.squ.edu.om/fedora/linux/releases/13/Everything/i386/debug
http://mirror.squ.edu.om/fedora/linux/releases/13/Everything/i386/debug/r...
http://mirror.squ.edu.om/fedora/linux/releases/13/Everything/i386/os
http://mirror.squ.edu.om/fedora/linux/releases/13/Everything/i386/os/Pack...
http://mirror.squ.edu.om/fedora/linux/releases/13/Everything/i386/os/drpms
Ending crawl 2011-11-02T07:56:58.109368
Starting crawl 2011-11-02T15:00:23.247313
http://mirror.squ.edu.om/fedora/linux/development/16/i386/os
http://mirror.squ.edu.om/fedora/linux/development/16/i386/os/drpms
http://mirror.squ.edu.om/fedora/linux/development/16/i386/os/images
http://mirror.squ.edu.om/fedora/linux/development/16/x86_64/os
http://mirror.squ.edu.om/fedora/linux/development/16/x86_64/os/EFI/BOOT
http://mirror.squ.edu.om/fedora/linux/development/16/x86_64/os/drpms
Directory pub/fedora/linux/development/rawhide/x86_64/debug is not up-to-date on this
host.
Directory pub/fedora/linux/development/rawhide/i386/debug is not up-to-date on this host.
Directory pub/fedora/linux/development/16/x86_64/os/images is not up-to-date on this
host.
Directory pub/fedora/linux/development/16/source/SRPMS/repodata is not up-to-date on this
host.
Directory pub/fedora/linux/development/16/i386/os/Packages is not up-to-date on this
host.
Directory pub/fedora/linux/development/16/source/SRPMS is not up-to-date on this host.
Directory pub/fedora/linux/development/rawhide/source/SRPMS/repodata is not up-to-date on
this host.
Directory pub/fedora/linux/development/rawhide/i386/debug/repodata is not up-to-date on
this host.
Directory pub/fedora/linux/development/16/i386/debug/repodata is not up-to-date on this
host.
Directory pub/fedora/linux/development/16/i386/os/images/pxeboot is not up-to-date on this
host.
Directory pub/fedora/linux/development/16/x86_64/os/isolinux is not up-to-date on this
host.
Directory pub/fedora/linux/development/rawhide/x86_64/debug/repodata is not up-to-date on
this host.
Directory pub/fedora/linux/development/16/i386/os/isolinux is not up-to-date on this
host.
Directory pub/fedora/linux/development/16/x86_64/os/repodata is not up-to-date on this
host.
Directory pub/fedora/linux/development/16/i386/debug is not up-to-date on this host.
Directory pub/fedora/linux/development/rawhide/i386/os/Packages is not up-to-date on this
host.
Directory pub/fedora/linux/development/rawhide/source/SRPMS is not up-to-date on this
host.
Directory pub/fedora/linux/development/16/x86_64/debug/repodata is not up-to-date on this
host.
Directory pub/fedora/linux/development/16/x86_64/debug is not up-to-date on this host.
Directory pub/fedora/linux/development/rawhide/i386/os/drpms is not up-to-date on this
host.
Directory pub/fedora/linux/development/16/x86_64/os/Packages is not up-to-date on this
host.
Directory pub/fedora/linux/development/16/i386/os/repodata is not up-to-date on this
host.
Directory pub/fedora/linux/development/16/x86_64/os/images/pxeboot is not up-to-date on
this host.
Directory pub/fedora/linux/development/rawhide/x86_64/os/Packages is not up-to-date on
this host.
Directory pub/fedora/linux/development/rawhide/i386/os/repodata is not up-to-date on this
host.
Ending crawl 2011-11-02T15:21:05.468727
--
Matt Domsch
Technology Strategist
Dell | Office of the CTO
-----Original Message-----
From: SQU Mirror Server [mailto:mirror@squ.edu.om]
Sent: Wednesday, November 02, 2011 5:39 AM
To: Domsch, Matt; mirror-admin(a)fedoraproject.org
Subject: RE: Request to add our mirror to your database
squ mirror server ip address is 62.231.244.17 ( Oman) my ip address: 62.231.235.7 (
Oman)
MM gives me this ftp.tsukuba.wide.ad.jp (203.178.132.86 )
__________________________________
Mirror Admin Team
Center for Information Systems,
Sultan Qaboos University,
P.O. Box 40, Postal Code 123,
Sultanate of Oman.
CIS Telephone number: (+968)24141500
CIS Fax Number: (+968)24413320
________________________________________
From: Matt_Domsch(a)Dell.com [Matt_Domsch(a)Dell.com]
Sent: Tuesday, November 01, 2011 11:58 PM
To: SQU Mirror Server; mirror-admin(a)fedoraproject.org
Subject: RE: Request to add our mirror to your database
What IP address are you seen coming from (e.g. the public IP address) ? Normally MM would
hand back a mirror that is geographically close, certainly within Japan where we have
several mirrors...
--
Matt Domsch
Technology Strategist
Dell | Office of the CTO
-----Original Message-----
From: mirror-admin-bounces(a)fedoraproject.org
[mailto:mirror-admin-bounces@fedoraproject.org] On Behalf Of SQU Mirror Server
Sent: Tuesday, November 01, 2011 9:19 AM
To: Domsch, Matt; mirror-admin(a)fedoraproject.org
Subject: RE: Request to add our mirror to your database
Dear Adrian,
It seems that our mirror server works fine so far but I notice that when I try to download
Fedora iso, fedora website gives me a mirror server which is located so far from my
country such as Japan. I think it should suggest to me a nearest mirror server to me, in
my case Oman. Is this a normal thing or there is a mistake some where ?
Kind regards,
Fahad Al-saidi
__________________________________
Mirror Admin Team
Center for Information Systems,
Sultan Qaboos University,
P.O. Box 40, Postal Code 123,
Sultanate of Oman.
CIS Telephone number: (+968)24141500
CIS Fax Number: (+968)24413320
________________________________________
From: Matt_Domsch(a)Dell.com [Matt_Domsch(a)Dell.com]
Sent: Friday, September 30, 2011 7:21 PM
To: SQU Mirror Server; mirror-admin(a)fedoraproject.org
Subject: RE: Request to add our mirror to your database
In Fedora, you set up and manage your own mirror information in our database, called
MirrorManager. Please see below.
-----Original Message-----
From: mirror-admin-bounces(a)fedoraproject.org
[mailto:mirror-admin-bounces@fedoraproject.org] On Behalf Of Adrian Reber
Sent: Monday, September 19, 2011 8:19 AM
To: SQU Mirror Server
Cc: fahad.alsaidi(a)ita.gov.om; mirror-admin(a)fedoraproject.org; ssdcis
Subject: Re: Request to add our mirror to your database
On Sat, Sep 17, 2011 at 10:34:39AM +0400, SQU Mirror Server wrote:
We are at Sultan Qaboos University finalizing the process of creating
a public mirror for FEDORA.
As per the instructions in your website, please find the required
details below:
IP address: 62.231.244.17
Location (country): Oman.
Outbound bandwidth: 155 Mb/s.
Thanks for setting up a Fedora mirror to get it listed please create an account in the
MirrorManager[2]; register your mirror in the mirror manager; enter your local netblocks
in the mirror manager (if desired); run report_mirror[3] and as soon as you are listed in
the database as up to date yum (and other clients) will be redirected to your mirror.
All this information can be found in more details in the wiki at:
http://fedoraproject.org/wiki/Infrastructure/Mirroring
Thanks for supporting Fedora!
If there are any more question do not hesitate to ask.
Adrian
[1]
http://fedoraproject.org/wiki/Infrastructure/Mirroring/Tiering
[2]
http://fedoraproject.org/wiki/Infrastructure/MirrorManager
[3]
http://fedoraproject.org/wiki/Infrastructure/Mirroring
--
Matt Domsch
Technology Strategist
Dell | Office of the CTO
From: mirror-admin-bounces(a)fedoraproject.org
[mailto:mirror-admin-bounces@fedoraproject.org] On Behalf Of SQU Mirror Server
Sent: Monday, September 26, 2011 4:48 AM
To: mirror-admin(a)fedoraproject.org
Subject: FW: Request to add our mirror to your database
Reminder
________________________________
From: SQU Mirror Server
Sent: Monday, September 12, 2011 8:25 AM
To: mirrors-subscribe(a)apache.org
Cc: fahad.alsaidi(a)ita.gov.om; ssdcis
Subject: Request to add our mirror to your database Dear APACHE Admin team, We are at
Sultan Qaboos University finalizing the process of creating a public mirror for APACHE.
Below is the URL to Apache:
The server address
http://mirror.squ.edu.om/apache/
Thanks,
__________________________________
Mirror Admin Team
Center for Information Systems,
Sultan Qaboos University,
P.O. Box 40, Postal Code 123,
Sultanate of Oman.
CIS Telephone number: (+968)24141500
CIS Fax Number: (+968)24413320
_______________________________________________
Mirror-admin mailing list
Mirror-admin(a)fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/mirror-admin