i have problem in crawel
by 顾刘健
i set a new site on mirrormanager, and also i have report_mirror run successful ,but its crawel always told me that Ending crawl of <Host(2268 - mirrors.njupt.edu.cn)> with status 5,is there any help for me? what shows in log is
2017-04-27 17:20:44,949 - INFO - Worker '3a08f8e' starting on host <Host(2268 - mirrors.njupt.edu.cn)>
2017-04-27 17:21:01,346 - INFO - Crawling with URL None
2017-04-27 17:21:01,471 - INFO - Ending crawl of <Host(2268 - mirrors.njupt.edu.cn)> with status 5
2017-04-28 17:47:52,276 - INFO - Worker '2630be2' starting on host <Host(2268 - mirrors.njupt.edu.cn)>
2017-04-28 17:47:52,353 - INFO - Crawling with URL None
2017-04-28 17:47:52,408 - INFO - Ending crawl of <Host(2268 - mirrors.njupt.edu.cn)> with status 5
6 years, 1 month
MirrorManager documentation and links
by Brian Long (brilong)
I started getting output from report_mirror saying exception ProtocolError not defined. I had to dig around to find the latest client code and compare to what I had installed. I wonder if some links need to be cleaned up on the fedoraproject.org website.
In my mind, the following site is the source of truth wrt. MirrorManager:
https://fedoraproject.org/wiki/Infrastructure/Mirroring
I think some of the links on this page might need to be updated. Specifically, where it says “The MirrorManager software keeps track of all the mirrors”, http://mirrormanager.org/ takes me to https://apps.fedoraproject.org/ and I have to manually select MirrorManager from the graph. I finally get directed to https://admin.fedoraproject.org/mirrormanager/. It would be nice if http://mirrormanager.org/ took me straight to the right URL.
At the bottom of the https://admin.fedoraproject.org/mirrormanager/ page, it has a link to https://fedorahosted.org/mirrormanager/ which is broken since fedorahosted is retired. I then manually click on https://pagure.io/ and have to search for mirrormanager. I get directed here: https://pagure.io/mirrormanager. The Overview tab needs to be reformatted and it’s specific to MirrorManager server.
I ended up downloading https://pagure.io/mirrormanager/blob/master/f/client/report_mirror and comparing to the version I had installed (mirrormanager-client-1.4.4-6.el7.noarch). It turns out the only difference is as follows:
--- /usr/bin/report_mirror 2016-06-23 09:55:20.000000000 -0400
+++ /usr/local/bin/report_mirror 2017-04-13 07:45:26.059404424 -0400
@@ -290,7 +292,7 @@
print server.checkin(data)
except socket.error, m:
print "Error checking in: %s. Please try again later." % (m[1])
- except ProtocolError:
+ except xmlrpclib.ProtocolError:
print "Error checking in: Service Temporarily Unavailable. Please try again later."
sys.exit(1)
except xmlrpclib.Fault:
Also, what’s the difference between the code on pagure.io and https://github.com/fedora-infra/mirrormanager2/? This github link has README containing old mailing list information:
Mailing list for announcements:http://www.redhat.com/mailman/listinfo/mirror-list
Mailing list for discussions:http://www.redhat.com/mailman/listinfo/mirror-list-d
Could someone update the mirrormanager-client to include the new code? Could someone consider changing some of the links I mentioned above?
Thank you.
/Brian/
--
Brian Long
Cisco Systems, Inc.
Government, Trust and Technology Services
Security & Trust Organization
Research Triangle Park, NC
6 years, 1 month
q-f-m or master fullfiletimelist problem?
by Chuck Anderson
I have one system I keep pointed directly at my mirror (not relying on
mirrorlist) so I can find problems with my local mirror. Today it says:
Cannot download 'http://download.wpi.edu/pub/fedora/linux/updates/25/x86_64/': Yum repo downloading error: Downloading error(s): repodata/0cbc1e966823acb0f4addccb4db219617335c54e85554fed2f3bca01c6383ea4-filelists.xml.gz - Cannot download, all mirrors were already tried without success; repodata/3e93bf5f3f9f9e242a136cd8487f9b917880373791533ec41c429f3942d1b61f-prestodelta.xml.xz - Cannot download, all mirrors were already tried without success; repodata/32d7de1743b6533476c1dfbcc9a7ded8fb71cf389b21e61602ede8313559d5dc-primary.xml.gz - Cannot download, all mirrors were already tried without success; repodata/fc618e356dcf2d17199eb8118651274f146c4bb4720b1b9ab8a731036daed7da-updateinfo.xml.xz - Cannot download, all mirrors were already tried without success.
Error: Failed to synchronize cache for repo 'updates'
I traced this problem to a stale repomd.xml from Apr 3:
-rw-r--r--. 1 mirror mirror 4848 Apr 3 18:12 repomd.xml
Compare this to the one on the master:
-rw-r--r-- 4848 2017/04/05 14:27:41 repomd.xml
Same size. Uh oh. q-f-m doesn't think this file needs to be
synced--it isn't showing up in the mastertransferlist:
>grep updates/25/x86_64/repodata/repomd.xml quick-mirror.YksUUJhBIF/*
quick-mirror.YksUUJhBIF/fullfiletimelist-fedora:1491369612 f 4848 linux/updates/25/x86_64/repodata/repomd.xml
quick-mirror.YksUUJhBIF/fullfiletimelist-fedora:e2fcb793861a193d2c3d8ed255723bfa241c2a34 linux/updates/25/x86_64/repodata/repomd.xml
>sha1sum /srv/ftp/pub/fedoraproject.org/fedora/linux/updates/25/x86_64/repodata/repomd.xml ~/repomd.xml
5145e1fcb2207cce33a0d202e6834b5fe792a3e1 /srv/ftp/pub/fedoraproject.org/fedora/linux/updates/25/x86_64/repodata/repomd.xml
d76ad1ef03095a97e2d5e4c9a36cf4598c4b4477 /home/cra/repomd.xml
(the last file being one I directly downloaded from the master just now)
6 years, 1 month
Troubles with the Fedora mirror at mirror.umd.edu
by Adrian Reber
We had reports that the Fedora mirror mirror.umd.edu is hanging for some
users. I can also see that the last rsync scan of your Fedora mirror
took over 6 hours. Somethings seem to hang or are very slow.
I disabled your Fedora mirror for now. Once you had a chance to look if
there are any problems you can re-enabled your mirror in MirrorManager
again or let me know, I can also re-enable it again.
Thanks for supporting Fedora.
Adrian
Connecting to https://mirror.umd.edu/ with a browser gives a warning as
some parts are still transmitted via HTTP. This is unrelated to the
Fedora mirror.
6 years, 1 month
Re: q-f-m or master fullfiletimelist problem?
by Chuck Anderson
After my last email, this resolved itself, but it has now happened
again... The latest repomd.xml I have is from April 6 even though I've
been running q-f-m every 15 minutes...
On Wed, Apr 05, 2017 at 03:05:06PM -0400, Chuck Anderson wrote:
> I have one system I keep pointed directly at my mirror (not relying on
> mirrorlist) so I can find problems with my local mirror. Today it says:
>
> Cannot download 'http://download.wpi.edu/pub/fedora/linux/updates/25/x86_64/': Yum repo downloading error: Downloading error(s): repodata/0cbc1e966823acb0f4addccb4db219617335c54e85554fed2f3bca01c6383ea4-filelists.xml.gz - Cannot download, all mirrors were already tried without success; repodata/3e93bf5f3f9f9e242a136cd8487f9b917880373791533ec41c429f3942d1b61f-prestodelta.xml.xz - Cannot download, all mirrors were already tried without success; repodata/32d7de1743b6533476c1dfbcc9a7ded8fb71cf389b21e61602ede8313559d5dc-primary.xml.gz - Cannot download, all mirrors were already tried without success; repodata/fc618e356dcf2d17199eb8118651274f146c4bb4720b1b9ab8a731036daed7da-updateinfo.xml.xz - Cannot download, all mirrors were already tried without success.
> Error: Failed to synchronize cache for repo 'updates'
>
> I traced this problem to a stale repomd.xml from Apr 3:
>
> -rw-r--r--. 1 mirror mirror 4848 Apr 3 18:12 repomd.xml
>
> Compare this to the one on the master:
>
> -rw-r--r-- 4848 2017/04/05 14:27:41 repomd.xml
>
> Same size. Uh oh. q-f-m doesn't think this file needs to be
> synced--it isn't showing up in the mastertransferlist:
>
> >grep updates/25/x86_64/repodata/repomd.xml quick-mirror.YksUUJhBIF/*
> quick-mirror.YksUUJhBIF/fullfiletimelist-fedora:1491369612 f 4848 linux/updates/25/x86_64/repodata/repomd.xml
> quick-mirror.YksUUJhBIF/fullfiletimelist-fedora:e2fcb793861a193d2c3d8ed255723bfa241c2a34 linux/updates/25/x86_64/repodata/repomd.xml
>
> >sha1sum /srv/ftp/pub/fedoraproject.org/fedora/linux/updates/25/x86_64/repodata/repomd.xml ~/repomd.xml
> 5145e1fcb2207cce33a0d202e6834b5fe792a3e1 /srv/ftp/pub/fedoraproject.org/fedora/linux/updates/25/x86_64/repodata/repomd.xml
> d76ad1ef03095a97e2d5e4c9a36cf4598c4b4477 /home/cra/repomd.xml
>
> (the last file being one I directly downloaded from the master just now)
6 years, 2 months
Re: fedora.westmancom.com
by Graham Johnston
If you add me as a mirror admin I'd appreciate that. My username is johnstong with email johnstong(a)westmancom.com.
Thanks,
Graham
On Apr 5, 2017 12:54 PM, Adrian Reber <adrian(a)lisas.de> wrote:
On Wed, Apr 05, 2017 at 03:25:32PM +0000, Graham Johnston wrote:
> I am from Westman Communications Group. We currently operate an EPEL mirror at fedora.westmancom.com. I've got a recollection from years ago that there was some sort of web admin portal for the mirror, and if there is, I've somehow lost control of it from the standpoint of maintaining the details in it on the Fedora side of things. I'm just trying to get the mirror back into shape right now as we are experiencing a synchronization error.
>
> You should be able to verify my authority by way of ARIN records:
> https://whois.arin.net/rest/org/WMAN
> https://whois.arin.net/rest/org/WMAN/pocs
> https://whois.arin.net/rest/poc/JOHNS39-ARIN.html
The mirror admin in our web admin portal (MirrorManager) is 'westman'
with the following IP address: pierceb(a)westmancom.com
If this person is you (or around) this account can be used to make the
necessary changes or add additional mirror admins to your mirror.
I can also add you as an mirror admin if you have an account in the
Fedora account system (FAS).
Adrian
6 years, 2 months
Fedora mirror at mirrors.nic.cz is not up to date
by Adrian Reber
We had reports that your mirror (mirrors.nic.cz) is no longer up to
date. Usually this would not be a problem as MirrorManager is disabling
outdated mirrors, but as you are also using report_mirror which claims
that your mirror is up to date, the state of the mirror is flipping.
It seems parts (especially fedora-secondary) of your mirror are no
longer updated since 2017-04-02. Could please have a look if something
is not working correctly. Thanks for supporting Fedora with your mirror.
I have temporarily disabled your mirrors as it seems to be a problem for
some clients. You can re-enabled it yourself or let me know.
Adrian
6 years, 2 months
