On Thu, Jan 28, 2016 at 08:41:19AM -0500, Matthew Miller wrote:
I'm getting 404 errors from several places to which mirrormanager
wants
to send me to get
https://download.fedoraproject.org/pub/alt/atomic/stable/Cloud-Images/x86...
(Specifically,
mirrors.rit.edu and
mirrors.kernel.org.) Presumbably,
these just haven't synced yet. But doesn't mirrormanager get feedback
about when mirrors are fresh?
I can try to give a few reasons (excuses) why you were redirected to a
wrong file.
* The file does not exist anymore on the master mirror. Not being 100%
sure how the file names are constructed but it seems to include a date
and the file on the master mirror (and a few other mirrors includes a
'.2'). So it seems like the file was recreated. Using the new file
name I am redirected to a mirror which has the file.
* /pub/alt (also known as the MM category 'Fedora Other') on the master
mirror is only scanned once per day (12:15 UTC). As the scanning
takes a very long time and as it creates a high I/O load the less
important MM categories like 'Fedora Other' and 'Fedora Archive' are
only scanned once a day.
*
mirrors.rit.edu and
mirrors.kernel.org are both running report_mirror
to update the status of their mirror after the sync. We trust mirrors
running report_mirror. We only get the information that they claim
the transmitted directories are up to date. The actual content is not
checked when using report_mirror. In the current case both mirrors
have been crawled by since the last report_mirror, but at the time of
the crawl the file 'Fedora-Cloud-Atomic-23-20160127.2.x86_64.qcow2'
(with the '.2') was not yet part of the database as it has a time of
'Jan 27 17:29' which is after the previous master mirror scan. By now
the new file should be part of the database and soon the crawler
should detect it also on the mirrors.
So there are many reasons why it did fail but the main reason probably
is, that due to large amount of files it is difficult to have real-time
view of what is on the mirrors and the master mirror. Finding the right
balance between massive amounts of I/O on the master and mirrors and
freshness of the MM database is the hard part.
Adrian