haproxy and mirrorlists

[Stephen John Smoogen]

Wow.. I thought this was already in place a long time ago. I say this
looks good and should be done.

On 28 May 2015 at 16:59, Kevin Fenzi <kevin at scrye.com> wrote:
> So, currently we have 5 mirrorlists servers, spread out around the
> world in 5 datacenters.
>
> Haproxy on our proxies sends requests to all 5 of those as they come
> in. This means for example that a request could hit proxy01 and it
> could send it to mirrorlist-host1plus in germany and then the reply
> comes back out proxy01 to the user. Not only does that mean there's
> more traffic on our vpn, but the answer is slower for the user.
>
> 5 of our datacenters have a local mirrorlist in the same datacenter,
> but currently it's just one of 5, so gets only 20% of traffic or so.
>
> I'd like to change the haproxy config from being a file to a template.
> In that mirrorlist section:
>
> listen  mirror-lists 0.0.0.0:10002
>     balance hdr(appserver)
>     timeout connect 30s
> {% if datacenter == 'dedicatedsolutions' %}
>     server  mirrorlist-dedicatedsolutions mirrorlist-dedicatedsolutions:80 check inter 5s rise 2 fall 3
>     server  mirrorlist-host1plus mirrorlist-host1plus:80 check inter 5s rise 2 fall 3 backup
>     server  mirrorlist-ibiblio mirrorlist-ibiblio:80 check inter 5s rise 2 fall 3 backup
>     server  mirrorlist-osuosl mirrorlist-osuosl:80 check inter 5s rise 2 fall 3 backup
>     server  mirrorlist-phx2 mirrorlist-phx2:80 check inter 5s rise 2 fall 3 backup
> {% elif datacenter == 'host1plus' %}
>     server  mirrorlist-dedicatedsolutions mirrorlist-dedicatedsolutions:80 check inter 5s rise 2 fall 3 backup
>     server  mirrorlist-host1plus mirrorlist-host1plus:80 check inter 5s rise 2 fall 3
>     server  mirrorlist-ibiblio mirrorlist-ibiblio:80 check inter 5s rise 2 fall 3 backup
>     server  mirrorlist-osuosl mirrorlist-osuosl:80 check inter 5s rise 2 fall 3 backup
>     server  mirrorlist-phx2 mirrorlist-phx2:80 check inter 5s rise 2 fall 3 backup
> {% elif datacenter == 'ibiblio' %}
>     server  mirrorlist-dedicatedsolutions mirrorlist-dedicatedsolutions:80 check inter 5s rise 2 fall 3 backup
>     server  mirrorlist-host1plus mirrorlist-host1plus:80 check inter 5s rise 2 fall 3 backup
>     server  mirrorlist-ibiblio mirrorlist-ibiblio:80 check inter 5s rise 2 fall 3
>     server  mirrorlist-osuosl mirrorlist-osuosl:80 check inter 5s rise 2 fall 3 backup
>     server  mirrorlist-phx2 mirrorlist-phx2:80 check inter 5s rise 2 fall 3 backup
> {% elif datacenter == 'osuosl' %}
>     server  mirrorlist-dedicatedsolutions mirrorlist-dedicatedsolutions:80 check inter 5s rise 2 fall 3 backup
>     server  mirrorlist-host1plus mirrorlist-host1plus:80 check inter 5s rise 2 fall 3 backup
>     server  mirrorlist-ibiblio mirrorlist-ibiblio:80 check inter 5s rise 2 fall 3 backup
>     server  mirrorlist-osuosl mirrorlist-osuosl:80 check inter 5s rise 2 fall 3
>     server  mirrorlist-phx2 mirrorlist-phx2:80 check inter 5s rise 2 fall 3 backup
> {% elif datacenter == 'phx2' %}
>     server  mirrorlist-dedicatedsolutions mirrorlist-dedicatedsolutions:80 check inter 5s rise 2 fall 3 backup
>     server  mirrorlist-host1plus mirrorlist-host1plus:80 check inter 5s rise 2 fall 3 backup
>     server  mirrorlist-ibiblio mirrorlist-ibiblio:80 check inter 5s rise 2 fall 3 backup
>     server  mirrorlist-osuosl mirrorlist-osuosl:80 check inter 5s rise 2 fall 3 backup
>     server  mirrorlist-phx2 mirrorlist-phx2:80 check inter 5s rise 2 fall 3
> {% else %}
>     server  mirrorlist-dedicatedsolutions mirrorlist-dedicatedsolutions:80 check inter 5s rise 2 fall 3
>     server  mirrorlist-host1plus mirrorlist-host1plus:80 check inter 5s rise 2 fall 3
>     server  mirrorlist-ibiblio mirrorlist-ibiblio:80 check inter 5s rise 2 fall 3
>     server  mirrorlist-osuosl mirrorlist-osuosl:80 check inter 5s rise 2 fall 3
>     server  mirrorlist-phx2 mirrorlist-phx2:80 check inter 5s rise 2 fall 3
> {% endif %}
>     option  httpchk GET /mirrorlist
>     option  allbackups
>
> This means that for the 5 proxies with a local mirrorlist server in their same datacenter,
> they use that one unless it's down, then they use any of the other ones.
>
> This should reduce traffic on our vpn and give people faster mirror answers.
>
> Did I miss anything here? Does the change look good?
>
> kevin
>
> _______________________________________________
> infrastructure mailing list
> infrastructure at lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/infrastructure



-- 
Stephen J Smoogen.