repo-mirrorlist quality control?

Stephen John Smoogen smooge at gmail.com
Wed Feb 25 21:16:27 UTC 2015


On 25 February 2015 at 10:58, Ralf Corsepius <rc040203 at freenet.de> wrote:

> On 02/25/2015 06:22 PM, Kevin Fenzi wrote:
>
>> On Wed, 25 Feb 2015 17:47:47 +0100
>> Ralf Corsepius <rc040203 at freenet.de> wrote:
>>
>
>  In another (much smaller) project, when pushing updates, we had
>>> removed all mirrors from our mirrorlists and had let the server poll
>>> "repodata/repomd.xml" from the mirrors to re-adde a mirror to the
>>> mirrorlists if this file matched. Of course, this is a primitive
>>> heuristic, but it had worked quite well for us.
>>>
>>
>> yeah, sadly we don't have any ability to push to mirrors, they have to
>> pull from us.
>>
> I sense a misunderstanding. With "pushing" I was referring to
> "uploading/publishing" a package to the repository on the master.
> The mirrors were polling ad lib, at intervals at their preference.
>
> I.e. when updating the master repository on the server, we emptied our
> mirrorlists, and let the _master_ poll repomd.xmls from the _mirrors_. When
> it found a mirror's "repomd.xml" was in sync, the mirrors was re-add to our
> mirrorlists on the server.
>
>  The mirrormanager crawler pulls repomd.xml from each
>> mirror when it crawls and compares it. However, as noted we can't poll
>> all mirrors all the time.
>>
> This sounds pretty much like the same approach we had taken.
>
>
>  It is a provable matter of fact that it points users (yum,
>>>>> mock, ...) to broken and out of sync mirrors.
>>>>>
>>>>
>>>> There will be such times, sure.
>>>>
>>> Here (Germany), in recent times, they happen almost daily. My guess
>>> is the "fedora flavors", the launch of f22 and the mass-rebuild on
>>> rawhide are showing their nasty side.
>>>
>>
>> So, to be clear you see daily where 'yum update' gives you all mirrors
>> erroring out and you cannot get a update list? And 'yum clean all'
>> doesn't help?
>>
> As I am heavily using mock, I am fairly often seeing this issue with mock.
> With mock w/ rawhide I am even observing hard build-breakdowns (Seems to me
> as if something in f21's mock was changed to not let it use mirrors)
>
>
My first impulse would be to look at mirroring things closer to the site to
see if the problems go away. It would also cut down on your sites network
costs if it was pulled down once versus for all the builds you need to do.

My second item would be to get an idea of what list of mirrors that are
being seen broken, and what way mirrormanager is serving them to clients.
Mirrormanager tries to figure out what systems are 'closest' to a location
to make speeds 'faster'. Since we aren't getting reports of this from all
over Europe but only certain segments I am wanting to see if the mirrors in
your network route or geolocation are stuck for some reason.

As Kevin listed we do a similar method to what you outlined.. the issue is
that there are a LOT of mirrors and there are a LOT of files which have to
be checked to make sure a mirror is not just up to date but complete. A
mirror might have gotten the repoxml file but not the rpms which are listed
in it. Another might have the rpms for some of the repository but for some
reason not all the files at that time. When the crawler goes through each
mirror it has to figure out all the ways that mirrors do things.

[In other words, be glad it works at any time... ]




>  The next time this happens can you file a ticket with the output?
>>
> I can try - However, during some periods in recent weeks, these incidents
> were so frequent, I'd not to anything else but filing tickets ;)
>
>
Doing it just once would be enough to start tracking it down.



-- 
Stephen J Smoogen.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fedoraproject.org/pipermail/devel/attachments/20150225/7e261bc4/attachment.html>


More information about the devel mailing list