A quick look at the yum mirrorlists

[Nicholas Miell]

After getting sick of yum trying to parse 404 Not Found error HTML as
repomd.xml (btw, Seth: yum will try to parse 404 Not Found error HTML as
repomd.xml), I used a quick mirror validation script of my own creation
(see below) on the official FC 4 mirror lists and determined that a lot
of the mirrors in that list really shouldn't be there.

http://dl.atrpms.net/mirrors/fedoracore/4/ppc/os/
	HTTP/1.1 404 Not Found
http://dl.atrpms.net/mirrors/fedoracore/updates/4/ppc/
	HTTP/1.1 404 Not Found
http://dl.atrpms.net/mirrors/fedoracore/updates/testing/4/ppc/
	HTTP/1.1 404 Not Found
http://fedora.arcticnetwork.ca/4/ppc/os/
	HTTP/1.1 404 Not Found
http://fedora.arcticnetwork.ca/updates/4/ppc/
	HTTP/1.1 404 Not Found
http://fedora.arcticnetwork.ca/updates/testing/4/ppc/
	HTTP/1.1 404 Not Found
http://fedora.ngi.it/updates/testing/4/i386/
	HTTP/1.1 404 Not Found
http://fedora.ngi.it/updates/testing/4/ppc/
	HTTP/1.1 404 Not Found
http://fedora.ngi.it/updates/testing/4/x86_64/
	HTTP/1.1 404 Not Found
http://fr.rpmfind.net/linux/fedora/core/4/ppc/os/
	HTTP/1.1 404 Not Found
http://fr.rpmfind.net/linux/fedora/core/updates/4/ppc/
	HTTP/1.1 404 Not Found
http://fr.rpmfind.net/linux/fedora/core/updates/testing/4/ppc/
	HTTP/1.1 404 Not Found
http://ftp1.skynet.cz/pub/linux/fedora/updates/testing/4/i386/
	HTTP/1.1 404 Not Found
http://ftp1.skynet.cz/pub/linux/fedora/updates/testing/4/ppc/
	HTTP/1.1 404 Not Found
http://ftp1.skynet.cz/pub/linux/fedora/updates/testing/4/x86_64/
	HTTP/1.1 404 Not Found
http://ftp.ale.org/pub/fedora/linux/core/4/i386/os/
	HTTP/1.1 404 Not Found
http://ftp.ale.org/pub/fedora/linux/core/4/ppc/os/
	HTTP/1.1 404 Not Found
http://ftp.ale.org/pub/fedora/linux/core/4/x86_64/os/
	HTTP/1.1 404 Not Found
http://ftp.ale.org/pub/fedora/linux/core/updates/4/i386/
	HTTP/1.1 404 Not Found
http://ftp.ale.org/pub/fedora/linux/core/updates/4/ppc/
	HTTP/1.1 404 Not Found
http://ftp.ale.org/pub/fedora/linux/core/updates/4/x86_64/
	HTTP/1.1 404 Not Found
http://ftp.ale.org/pub/fedora/linux/core/updates/testing/4/i386/
	HTTP/1.1 404 Not Found
http://ftp.ale.org/pub/fedora/linux/core/updates/testing/4/ppc/
	HTTP/1.1 404 Not Found
http://ftp.ale.org/pub/fedora/linux/core/updates/testing/4/x86_64/
	HTTP/1.1 404 Not Found
http://ftp.chg.ru/pub/Linux/fedora/core/4/i386/os/
	HTTP/1.1 404 Not Found
http://ftp.chg.ru/pub/Linux/fedora/core/4/ppc/os/
	HTTP/1.1 404 Not Found
http://ftp.chg.ru/pub/Linux/fedora/core/4/x86_64/os/
	HTTP/1.1 404 Not Found
http://ftp.chg.ru/pub/Linux/fedora/core/updates/4/i386/
	HTTP/1.1 404 Not Found
http://ftp.chg.ru/pub/Linux/fedora/core/updates/4/ppc/
	HTTP/1.1 404 Not Found
http://ftp.chg.ru/pub/Linux/fedora/core/updates/4/x86_64/
	HTTP/1.1 404 Not Found
http://ftp.chg.ru/pub/Linux/fedora/core/updates/testing/4/i386/
	HTTP/1.1 404 Not Found
http://ftp.chg.ru/pub/Linux/fedora/core/updates/testing/4/ppc/
	HTTP/1.1 404 Not Found
http://ftp.chg.ru/pub/Linux/fedora/core/updates/testing/4/x86_64/
	HTTP/1.1 404 Not Found
http://ftp.iasi.roedu.net/mirrors/fedora.redhat.com/core/4/i386/os/
	HTTP/1.1 403 Forbidden
http://ftp.iasi.roedu.net/mirrors/fedora.redhat.com/core/4/ppc/os/
	HTTP/1.1 404 Not Found
http://ftp.iasi.roedu.net/mirrors/fedora.redhat.com/core/4/x86_64/os/
	HTTP/1.1 404 Not Found
http://ftp.iasi.roedu.net/mirrors/fedora.redhat.com/core/updates/4/ppc/
	HTTP/1.1 404 Not Found
http://ftp.iasi.roedu.net/mirrors/fedora.redhat.com/core/updates/4/x86_64/
	HTTP/1.1 404 Not Found
http://ftp.iasi.roedu.net/mirrors/fedora.redhat.com/core/updates/testing/4/ppc/
	HTTP/1.1 404 Not Found
http://ftp.iasi.roedu.net/mirrors/fedora.redhat.com/core/updates/testing/4/x86_64/
	HTTP/1.1 404 Not Found
http://ftp.ipv6.funet.fi/pub/mirrors/ftp.redhat.com/pub/fedora/linux/core/4/i386/os/
	IPv6 only
http://ftp.ipv6.funet.fi/pub/mirrors/ftp.redhat.com/pub/fedora/linux/core/4/ppc/os/
	IPv6 only
http://ftp.ipv6.funet.fi/pub/mirrors/ftp.redhat.com/pub/fedora/linux/core/4/x86_64/os/
	IPv6 only
http://ftp.ipv6.funet.fi/pub/mirrors/ftp.redhat.com/pub/fedora/linux/core/updates/4/i386/
	IPv6 only
http://ftp.ipv6.funet.fi/pub/mirrors/ftp.redhat.com/pub/fedora/linux/core/updates/4/ppc/
	IPv6 only
http://ftp.ipv6.funet.fi/pub/mirrors/ftp.redhat.com/pub/fedora/linux/core/updates/4/x86_64/
	IPv6 only
http://ftp.ipv6.funet.fi/pub/mirrors/ftp.redhat.com/pub/fedora/linux/core/updates/testing/4/i386/
	IPv6 only
http://ftp.ipv6.funet.fi/pub/mirrors/ftp.redhat.com/pub/fedora/linux/core/updates/testing/4/ppc/
	IPv6 only
http://ftp.ipv6.funet.fi/pub/mirrors/ftp.redhat.com/pub/fedora/linux/core/updates/testing/4/x86_64/
	IPv6 only
http://ftp.lug.ro/fedora/linux/core/4/ppc/os/
	HTTP/1.1 404 Not Found
http://ftp.lug.ro/fedora/linux/core/updates/4/ppc/
	HTTP/1.1 404 Not Found
http://ftp.lug.ro/fedora/linux/core/updates/testing/4/ppc/
	HTTP/1.1 404 Not Found
http://ftp.tecnoera.com/pub/fedora/linux/core/4/ppc/os/
	HTTP/1.1 404 Not Found
http://ftp.tecnoera.com/pub/fedora/linux/core/4/x86_64/os/
	HTTP/1.1 404 Not Found
http://ftp.tecnoera.com/pub/fedora/linux/core/updates/testing/4/i386/
	HTTP/1.1 404 Not Found
http://ftp.tecnoera.com/pub/fedora/linux/core/updates/testing/4/ppc/
	HTTP/1.1 404 Not Found
http://ftp.tecnoera.com/pub/fedora/linux/core/updates/testing/4/x86_64/
	HTTP/1.1 404 Not Found
http://ftp.udl.es/pub/fedora/linux/core/4/ppc/os/
	HTTP/1.1 404 Not Found
http://ftp.udl.es/pub/fedora/linux/core/updates/4/ppc/
	HTTP/1.1 404 Not Found
http://ftp.udl.es/pub/fedora/linux/core/updates/testing/4/ppc/
	HTTP/1.1 404 Not Found
http://ftp.upjs.sk/pub/linux/fedora/linux/core/4/ppc/os/
	HTTP/1.1 404 Not Found
http://klid.dk/homeftp/fedora/linux/core/4/i386/os/
	HTTP/1.1 403 Forbidden
http://klid.dk/homeftp/fedora/linux/core/4/ppc/os/
	HTTP/1.1 403 Forbidden
http://klid.dk/homeftp/fedora/linux/core/4/x86_64/os/
	HTTP/1.1 403 Forbidden
http://klid.dk/homeftp/fedora/linux/core/updates/4/i386/
	HTTP/1.1 403 Forbidden
http://klid.dk/homeftp/fedora/linux/core/updates/4/ppc/
	HTTP/1.1 403 Forbidden
http://klid.dk/homeftp/fedora/linux/core/updates/4/x86_64/
	HTTP/1.1 403 Forbidden
http://klid.dk/homeftp/fedora/linux/core/updates/testing/4/i386/
	HTTP/1.1 403 Forbidden
http://klid.dk/homeftp/fedora/linux/core/updates/testing/4/ppc/
	HTTP/1.1 403 Forbidden
http://klid.dk/homeftp/fedora/linux/core/updates/testing/4/x86_64/
	HTTP/1.1 403 Forbidden
http://less.cogeco.net/pub/fedora/linux/core/4/ppc/os/
	HTTP/1.1 404 Not Found
http://less.cogeco.net/pub/fedora/linux/core/updates/4/ppc/
	HTTP/1.1 404 Not Found
http://less.cogeco.net/pub/fedora/linux/core/updates/testing/4/i386/
	HTTP/1.1 404 Not Found
http://less.cogeco.net/pub/fedora/linux/core/updates/testing/4/ppc/
	HTTP/1.1 404 Not Found
http://less.cogeco.net/pub/fedora/linux/core/updates/testing/4/x86_64/
	HTTP/1.1 404 Not Found
http://linux.admin.uillinois.edu/pub/linux/fedora/linux/core/4/i386/os/
	HTTP/1.1 403 Forbidden
http://linux.admin.uillinois.edu/pub/linux/fedora/linux/core/4/ppc/os/
	HTTP/1.1 403 Forbidden
http://linux.admin.uillinois.edu/pub/linux/fedora/linux/core/4/x86_64/os/
	HTTP/1.1 403 Forbidden
http://linux.admin.uillinois.edu/pub/linux/fedora/linux/core/updates/4/i386/
	HTTP/1.1 403 Forbidden
http://linux.admin.uillinois.edu/pub/linux/fedora/linux/core/updates/4/ppc/
	HTTP/1.1 403 Forbidden
http://linux.admin.uillinois.edu/pub/linux/fedora/linux/core/updates/4/x86_64/
	HTTP/1.1 403 Forbidden
http://linux.admin.uillinois.edu/pub/linux/fedora/linux/core/updates/testing/4/i386/
	HTTP/1.1 403 Forbidden
http://linux.admin.uillinois.edu/pub/linux/fedora/linux/core/updates/testing/4/ppc/
	HTTP/1.1 403 Forbidden
http://linux.admin.uillinois.edu/pub/linux/fedora/linux/core/updates/testing/4/x86_64/
	HTTP/1.1 403 Forbidden
http://mirror2etf.bg.ac.yu/distributions/fedora/4/i386/os/
	Host mirror2etf.bg.ac.yu not found: 3(NXDOMAIN)
http://mirror2etf.bg.ac.yu/distributions/fedora/4/ppc/os/
	Host mirror2etf.bg.ac.yu not found: 3(NXDOMAIN)
http://mirror2etf.bg.ac.yu/distributions/fedora/4/x86_64/os/
	Host mirror2etf.bg.ac.yu not found: 3(NXDOMAIN)
http://mirror2etf.bg.ac.yu/distributions/fedora/updates/4/i386/
	Host mirror2etf.bg.ac.yu not found: 3(NXDOMAIN)
http://mirror2etf.bg.ac.yu/distributions/fedora/updates/4/ppc/
	Host mirror2etf.bg.ac.yu not found: 3(NXDOMAIN)
http://mirror2etf.bg.ac.yu/distributions/fedora/updates/4/x86_64/
	Host mirror2etf.bg.ac.yu not found: 3(NXDOMAIN)
http://mirror2etf.bg.ac.yu/distributions/fedora/updates/testing/4/i386/
	Host mirror2etf.bg.ac.yu not found: 3(NXDOMAIN)
http://mirror2etf.bg.ac.yu/distributions/fedora/updates/testing/4/ppc/
	Host mirror2etf.bg.ac.yu not found: 3(NXDOMAIN)
http://mirror2etf.bg.ac.yu/distributions/fedora/updates/testing/4/x86_64/
	Host mirror2etf.bg.ac.yu not found: 3(NXDOMAIN)
http://mirror.averse.net/fedora/linux/core/4/ppc/os/
	HTTP/1.1 404 Not Found
http://mirror.averse.net/fedora/linux/core/4/x86_64/os/
	HTTP/1.1 404 Not Found
http://mirror.averse.net/fedora/linux/core/updates/4/ppc/
	HTTP/1.1 404 Not Found
http://mirror.averse.net/fedora/linux/core/updates/4/x86_64/
	HTTP/1.1 404 Not Found
http://mirror.averse.net/fedora/linux/core/updates/testing/4/i386/
	HTTP/1.1 404 Not Found
http://mirror.averse.net/fedora/linux/core/updates/testing/4/ppc/
	HTTP/1.1 404 Not Found
http://mirror.averse.net/fedora/linux/core/updates/testing/4/x86_64/
	HTTP/1.1 404 Not Found
http://mirror.netglobalis.net/pub/fedora/core/4/ppc/os/
	HTTP/1.1 404 Not Found
http://mirror.netglobalis.net/pub/fedora/core/updates/4/i386/
	HTTP/1.1 404 Not Found
http://mirror.netglobalis.net/pub/fedora/core/updates/4/ppc/
	HTTP/1.1 404 Not Found
http://mirror.netglobalis.net/pub/fedora/core/updates/4/x86_64/
	HTTP/1.1 404 Not Found
http://mirror.netglobalis.net/pub/fedora/core/updates/testing/4/i386/
	HTTP/1.1 404 Not Found
http://mirror.netglobalis.net/pub/fedora/core/updates/testing/4/ppc/
	HTTP/1.1 404 Not Found
http://mirror.netglobalis.net/pub/fedora/core/updates/testing/4/x86_64/
	HTTP/1.1 404 Not Found
http://mirrors.bevc.net/fedora/4/i386/os/
	HTTP/1.1 404 Not Found
http://mirrors.bevc.net/fedora/4/ppc/os/
	HTTP/1.1 404 Not Found
http://mirrors.bevc.net/fedora/4/x86_64/os/
	HTTP/1.1 404 Not Found
http://mirrors.bevc.net/fedora/updates/4/ppc/
	HTTP/1.1 404 Not Found
http://mirrors.playboy.com/fedora/4/ppc/os/
	HTTP/1.1 404 Not Found
http://mirrors.playboy.com/fedora/updates/4/ppc/
	HTTP/1.1 404 Not Found
http://mirrors.playboy.com/fedora/updates/testing/4/ppc/
	HTTP/1.1 404 Not Found
http://mirror.usu.edu/mirrors/fedora/linux/core/updates/4/i386/
	HTTP/1.1 404 Not Found
http://mirror.usu.edu/mirrors/fedora/linux/core/updates/4/ppc/
	HTTP/1.1 404 Not Found
http://mirror.usu.edu/mirrors/fedora/linux/core/updates/4/x86_64/
	HTTP/1.1 404 Not Found
http://srl.cs.jhu.edu/YUM/fedora-core/4/ppc/os/
	HTTP/1.1 404 Not Found
http://srl.cs.jhu.edu/YUM/fedora-core/updates/4/ppc/
	HTTP/1.1 404 Not Found
http://srl.cs.jhu.edu/YUM/fedora-core/updates/testing/4/ppc/
	HTTP/1.1 404 Not Found
http://www.fedora.is/fedora/core/4/ppc/os/
	HTTP/1.1 404 Not Found
http://zeniv.linux.org.uk/pub/distributions/fedora/linux/core/updates/testing/4/i386/
	HTTP/1.1 404 Not Found
http://zeniv.linux.org.uk/pub/distributions/fedora/linux/core/updates/testing/4/ppc/
	HTTP/1.1 404 Not Found
http://zeniv.linux.org.uk/pub/distributions/fedora/linux/core/updates/testing/4/x86_64/
	HTTP/1.1 404 Not Found


My verification script:

(for m in $(cat mirrorlist.txt); do
echo "$m $(curl -I -s ${m}repodata/repomd.xml | head -n 1)";
done) > mirrorstatus.txt
egrep -v "200 OK|302 Found" mirrorstatus.txt > badmirrors.txt

and then follow up with some manual result filtering.

-- 
Nicholas Miell <nmiell at comcast.net>