Unable to add HTTP url to mirror host in mirrormanager

[Samuel Patwin]

Hi Folks. My colleagues and I are doing some spring cleaning with our 
Mirror servers(mirror.rackspace.com) and I've noticed some 
self-inflicted decay with regards to our information as stored on the 
Fedora project so I'm working to get our mirrors properly set up in 
Mirror Manager and am running into some issues.

Firstly while I can add the https and rsync URLs to a host, I am unable 
to get our http url added to a host with a "Could not add Category URL 
to the host" error, likely because the url is already added to another 
host somewhere within MirrorManager. Looking through the mailing list 
archive it looks like my predecessors on-boarded our Mirrors using an 
FAS account "rackspace" which is now marked inactive(uses a dead email 
box). I'd like to get the URL freed up so I can add it to a Host that is 
on a Site tied to my team's active FAS accounts.

Additionally I'd like to find a way to properly have our mirrors added 
to the network while still taking advantage the "globally available" 
nature of our infrastructure. Our DNS entries are load balanced via F5 
GTM load balancers such that we use the same fqdn for our mirrors 
globally, but caching nameservers pull different entries depending on 
their latency/region. The problem is that we aren't able to add the same 
URL to multiple hosts as can be seen with my first issue. What I'd like 
to see is the ability for a single "Site" to have multiple hosts defined 
that are able to re-use the same URL in the case that global server load 
balancing via DNS is being used, possibly only with fedora admin 
approval and/or unique ASNs are used. If a better place for this second 
request is a feature request via github please just let me know and I'll 
be happy to file it.

Any assistance would be appreciated, thanks.

Sam P
Linux Patching
Rackspace

[Adrian Reber]

On Mon, Apr 01, 2019 at 04:02:02PM -0500, Samuel Patwin wrote:
>  Hi Folks. My colleagues and I are doing some spring cleaning with our
> Mirror
>  servers(mirror.rackspace.com) and I've noticed some self-inflicted decay
>  with regards to our information as stored on the Fedora project so I'm
>  working to get our mirrors properly set up in Mirror Manager and am running
>  into some issues.
>  
>  Firstly while I can add the https and rsync URLs to a host, I am unable to
>  get our http url added to a host with a "Could not add Category URL to the
>  host" error, likely because the url is already added to another host
>  somewhere within MirrorManager. Looking through the mailing list archive it
>  looks like my predecessors on-boarded our Mirrors using an FAS account
>  "rackspace" which is now marked inactive(uses a dead email box). I'd like
> to
>  get the URL freed up so I can add it to a Host that is on a Site tied to my
>  team's active FAS accounts. 
I deleted the old Site and Host with the HTTP URL which was owned by
the 'rackspace' account.

>  Additionally I'd like to find a way to properly have our mirrors
> added to
>  the network while still taking advantage the "globally available" nature of
>  our infrastructure. Our DNS entries are load balanced via F5 GTM load
>  balancers such that we use the same fqdn for our mirrors globally, but
>  caching nameservers pull different entries depending on their
>  latency/region. The problem is that we aren't able to add the same URL to
>  multiple hosts as can be seen with my first issue. What I'd like to see is
>  the ability for a single "Site" to have multiple hosts defined that are able
>  to re-use the same URL in the case that global server load balancing via DNS
>  is being used, possibly only with fedora admin approval and/or unique ASNs
>  are used. If a better place for this second request is a feature request via
>  github please just let me know and I'll be happy to file it. 
The correct place for a discussion like this would be github, indeed. I
understand why you want something like that, but the chances for this
being implemented are pretty low, as the URL currently is defined as a
unique value in the database and as our whole mirror decision process
kind of relies on the fact that it is unique. It sounds pretty
disruptive but let's discuss it on github.

		Adrian

[Samuel Patwin]

Adrian a delayed thanks for the assistance on this. Working with 
yourself and Matt via my opened github issue I've followed your guidance 
and gotten our dc-specific mirrors populated in MirrorManager as a 
workaround. To wrap this up I'd like to request that the following URLs 
be added back to the legacy "global" mirror list if possible:

http://mirror.rackspace.com/fedora
https://mirror.rackspace.com/fedora
rsync://mirror.rackspace.com/fedora
http://mirror.rackspace.com/epel
https://mirror.rackspace.com/epel
rsync://mirror.rackspace.com/epel

while these would be lowest-priority entries they would add to the 
global mirror "safety net" should a client be unable to reach any other 
country-specific mirrors. Thanks again for the assistance.

Sam P
Linux Patching
Rackspace

On 4/2/19 1:33 AM, Adrian Reber wrote:
>  On Mon, Apr 01, 2019 at 04:02:02PM -0500, Samuel Patwin wrote:
> > Hi Folks. My colleagues and I are doing some spring cleaning with our Mirror
> > servers(mirror.rackspace.com) and I've noticed some self-inflicted decay
> > with regards to our information as stored on the Fedora project so I'm
> > working to get our mirrors properly set up in Mirror Manager and am running
> > into some issues.
> >
> > Firstly while I can add the https and rsync URLs to a host, I am unable to
> > get our http url added to a host with a "Could not add Category URL to the
> > host" error, likely because the url is already added to another host
> > somewhere within MirrorManager. Looking through the mailing list archive it
> > looks like my predecessors on-boarded our Mirrors using an FAS account
> > "rackspace" which is now marked inactive(uses a dead email box). I'd
> like to
> > get the URL freed up so I can add it to a Host that is on a Site tied to my
> > team's active FAS accounts.
>  I deleted the old Site and Host with the HTTP URL which was owned by
>  the 'rackspace' account.
>
> > Additionally I'd like to find a way to properly have our mirrors added to
> > the network while still taking advantage the "globally available" nature
> of
> > our infrastructure. Our DNS entries are load balanced via F5 GTM load
> > balancers such that we use the same fqdn for our mirrors globally, but
> > caching nameservers pull different entries depending on their
> > latency/region. The problem is that we aren't able to add the same URL to
> > multiple hosts as can be seen with my first issue. What I'd like to see is
> > the ability for a single "Site" to have multiple hosts defined that are
> able
> > to re-use the same URL in the case that global server load balancing via DNS
> > is being used, possibly only with fedora admin approval and/or unique ASNs
> > are used. If a better place for this second request is a feature request via
> > github please just let me know and I'll be happy to file it.
>  The correct place for a discussion like this would be github, indeed. I
>  understand why you want something like that, but the chances for this
>  being implemented are pretty low, as the URL currently is defined as a
>  unique value in the database and as our whole mirror decision process
>  kind of relies on the fact that it is unique. It sounds pretty
>  disruptive but let's discuss it on github.
>
>  		Adrian

[Adrian Reber]

Sam,

good to hear you found a workaround, but I am sorry to say that I do not
know what you mean with: 'legacy "global" mirror'. I am not aware of
anything like that.

		Adrian

On Wed, Jul 17, 2019 at 03:42:06PM -0500, Samuel Patwin wrote:
>  Adrian a delayed thanks for the assistance on this. Working with
> yourself
>  and Matt via my opened github issue I've followed your guidance and gotten
>  our dc-specific mirrors populated in MirrorManager as a workaround. To wrap
>  this up I'd like to request that the following URLs be added back to the
>  legacy "global" mirror list if possible:
>  
>  http://mirror.rackspace.com/fedora
>  https://mirror.rackspace.com/fedora
>  rsync://mirror.rackspace.com/fedora
>  http://mirror.rackspace.com/epel
>  https://mirror.rackspace.com/epel
>  rsync://mirror.rackspace.com/epel
>  
>  while these would be lowest-priority entries they would add to the global
>  mirror "safety net" should a client be unable to reach any other
>  country-specific mirrors. Thanks again for the assistance.
>  
>  Sam P
>  Linux Patching
>  Rackspace
>  
>  On 4/2/19 1:33 AM, Adrian Reber wrote:
>  > On Mon, Apr 01, 2019 at 04:02:02PM -0500, Samuel Patwin wrote:
>  > > Hi Folks. My colleagues and I are doing some spring cleaning with our Mirror
>  > > servers(mirror.rackspace.com) and I've noticed some self-inflicted decay
>  > > with regards to our information as stored on the Fedora project so I'm
>  > > working to get our mirrors properly set up in Mirror Manager and am running
>  > > into some issues.
>  > > 
>  > > Firstly while I can add the https and rsync URLs to a host, I am unable to
>  > > get our http url added to a host with a "Could not add Category URL to
> the
>  > > host" error, likely because the url is already added to another host
>  > > somewhere within MirrorManager. Looking through the mailing list archive it
>  > > looks like my predecessors on-boarded our Mirrors using an FAS account
>  > > "rackspace" which is now marked inactive(uses a dead email box).
> I'd like to
>  > > get the URL freed up so I can add it to a Host that is on a Site tied to my
>  > > team's active FAS accounts.
>  > I deleted the old Site and Host with the HTTP URL which was owned by
>  > the 'rackspace' account.
>  > 
>  > > Additionally I'd like to find a way to properly have our mirrors added to
>  > > the network while still taking advantage the "globally available"
> nature of
>  > > our infrastructure. Our DNS entries are load balanced via F5 GTM load
>  > > balancers such that we use the same fqdn for our mirrors globally, but
>  > > caching nameservers pull different entries depending on their
>  > > latency/region. The problem is that we aren't able to add the same URL to
>  > > multiple hosts as can be seen with my first issue. What I'd like to see is
>  > > the ability for a single "Site" to have multiple hosts defined that
> are able
>  > > to re-use the same URL in the case that global server load balancing via DNS
>  > > is being used, possibly only with fedora admin approval and/or unique ASNs
>  > > are used. If a better place for this second request is a feature request via
>  > > github please just let me know and I'll be happy to file it.
>  > The correct place for a discussion like this would be github, indeed. I
>  > understand why you want something like that, but the chances for this
>  > being implemented are pretty low, as the URL currently is defined as a
>  > unique value in the database and as our whole mirror decision process
>  > kind of relies on the fact that it is unique. It sounds pretty
>  > disruptive but let's discuss it on github.
>  > 
>  > 		Adrian
>  
>  _______________________________________________
>  Mirror-admin mailing list -- mirror-admin(a)lists.fedoraproject.org
>  To unsubscribe send an email to mirror-admin-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]...

[Samuel Patwin]

Apologies Adrian, I was confused by the verbiage in 
https://github.com/fedora-infra/mirrormanager2/issues/194 in thinking 
that there was a separate list of global mirrors in Mirror Manager but 
reading more closely I presume references to the "global list" is just 
the sum total of all mirrors regardless of country? If that is the case 
then we should already be all set with our existing Mirror Manager host 
entries. Thanks.

Sam P
Linux Patching
Rackspace

On 7/18/19 1:40 AM, Adrian Reber wrote:
>  Sam,
>
>  good to hear you found a workaround, but I am sorry to say that I do not
>  know what you mean with: 'legacy "global" mirror'. I am not aware of
>  anything like that.
>
>  		Adrian
>
>  On Wed, Jul 17, 2019 at 03:42:06PM -0500, Samuel Patwin wrote:
> > Adrian a delayed thanks for the assistance on this. Working with yourself
> > and Matt via my opened github issue I've followed your guidance and gotten
> > our dc-specific mirrors populated in MirrorManager as a workaround. To wrap
> > this up I'd like to request that the following URLs be added back to the
> > legacy "global" mirror list if possible:
> >
> > http://mirror.rackspace.com/fedora
> > https://mirror.rackspace.com/fedora
> > rsync://mirror.rackspace.com/fedora
> > http://mirror.rackspace.com/epel
> > https://mirror.rackspace.com/epel
> > rsync://mirror.rackspace.com/epel
> >
> > while these would be lowest-priority entries they would add to the global
> > mirror "safety net" should a client be unable to reach any other
> > country-specific mirrors. Thanks again for the assistance.
> >
> > Sam P
> > Linux Patching
> > Rackspace
> >
> > On 4/2/19 1:33 AM, Adrian Reber wrote:
> >> On Mon, Apr 01, 2019 at 04:02:02PM -0500, Samuel Patwin wrote:
> >>> Hi Folks. My colleagues and I are doing some spring cleaning with our Mirror
> >>> servers(mirror.rackspace.com) and I've noticed some self-inflicted decay
> >>> with regards to our information as stored on the Fedora project so I'm
> >>> working to get our mirrors properly set up in Mirror Manager and am running
> >>> into some issues.
> >>>
> >>> Firstly while I can add the https and rsync URLs to a host, I am unable to
> >>> get our http url added to a host with a "Could not add Category URL to
> the
> >>> host" error, likely because the url is already added to another host
> >>> somewhere within MirrorManager. Looking through the mailing list archive it
> >>> looks like my predecessors on-boarded our Mirrors using an FAS account
> >>> "rackspace" which is now marked inactive(uses a dead email box).
> I'd like to
> >>> get the URL freed up so I can add it to a Host that is on a Site tied to my
> >>> team's active FAS accounts.
> >> I deleted the old Site and Host with the HTTP URL which was owned by
> >> the 'rackspace' account.
> >>
> >>> Additionally I'd like to find a way to properly have our mirrors added
> to
> >>> the network while still taking advantage the "globally available"
> nature of
> >>> our infrastructure. Our DNS entries are load balanced via F5 GTM load
> >>> balancers such that we use the same fqdn for our mirrors globally, but
> >>> caching nameservers pull different entries depending on their
> >>> latency/region. The problem is that we aren't able to add the same URL
> to
> >>> multiple hosts as can be seen with my first issue. What I'd like to see
> is
> >>> the ability for a single "Site" to have multiple hosts defined that
> are able
> >>> to re-use the same URL in the case that global server load balancing via DNS
> >>> is being used, possibly only with fedora admin approval and/or unique ASNs
> >>> are used. If a better place for this second request is a feature request via
> >>> github please just let me know and I'll be happy to file it.
> >> The correct place for a discussion like this would be github, indeed. I
> >> understand why you want something like that, but the chances for this
> >> being implemented are pretty low, as the URL currently is defined as a
> >> unique value in the database and as our whole mirror decision process
> >> kind of relies on the fact that it is unique. It sounds pretty
> >> disruptive but let's discuss it on github.
> >>
> >> 		Adrian
> > _______________________________________________
> > Mirror-admin mailing list -- mirror-admin(a)lists.fedoraproject.org
> > To unsubscribe send an email to mirror-admin-leave(a)lists.fedoraproject.org
> > Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]...