Setting up my own MirrorManager in air-gapped environment

[Kees Bakker]

Hi,

Sorry if this the wrong mailing list for this. Or should I create an issue on github?

I'm trying to setup my own MirrorManager [1]. The README gives two simple methods,
but none of them work for me. The Vagrant method only works when there is an IPA
server (ipa.tinystage.test) around. The other method (native, virtualenv) has lots of
issues. The setup description is close to useless.

Problems encountered so far:
* no example or description for client_secrets.json
* pip requirements needs itsdangerous==2.0.1
* flaskext is not ready for Python3
   File
"/home/keesb/src/mirrormanager2/.env/lib64/python3.10/site-packages/flaskext/xmlrpc.py",
line 252
     except Fault, fault:
            ^^^^^^^^^^^^
SyntaxError: multiple exception types must be parenthesized


So, the question is: how can I setup my own MirrorManager? Did anyone try this
recently?

[1] https://github.com/fedora-infra/mirrormanager2/
-- 
Kees

[Stephen John Smoogen]

On Thu, 6 Oct 2022 at 04:48, Kees Bakker <keesb(a)ghs.com&gt; wrote:

>  Hi,
>
>  Sorry if this the wrong mailing list for this. Or should I create an issue
>  on github?
>
>  
It may help to back up a bit and explain what the setup you are trying for.
Mirror manager is meant to deal with dozens of geolocated mirrors on the
internet and work out which ones are closest to the user and give a list of
some subset to them. It is also meant to allow for self-management of each
of the mirrors and then scanning of the sites to see if they are in sync.

If you aren't needing all of this, then mirrorlist may be overkill for what
you need. Air gapped networks generally do not have a way to give
geolocation to the site (as that would usually be a bad thing on an
air-gapped network). They are also usually more centrally managed.

The two sites I know which are using mirrormanager are Fedora and
RPMfusion. I expect that it has been set up in situ for a long time so
having it looked at by someone outside to figure out what isn't documented
is good.



>  I'm trying to setup my own MirrorManager [1]. The README gives
> two simple
>  methods,
>  but none of them work for me. The Vagrant method only works when there is
>  an IPA
>  server (ipa.tinystage.test) around. The other method (native, virtualenv)
>  has lots of
>  issues. The setup description is close to useless.
>
>  Problems encountered so far:
>  * no example or description for client_secrets.json
>  * pip requirements needs itsdangerous==2.0.1
>  * flaskext is not ready for Python3
>     File
>
> "/home/keesb/src/mirrormanager2/.env/lib64/python3.10/site-packages/flaskext/xmlrpc.py",
>  line 252
>       except Fault, fault:
>              ^^^^^^^^^^^^
>  SyntaxError: multiple exception types must be parenthesized
>
>
>  So, the question is: how can I setup my own MirrorManager? Did anyone try
>  this
>  recently?
>
>  [1] https://github.com/fedora-infra/mirrormanager2/
>  --
>  Kees
>  _______________________________________________
>  Mirror-admin mailing list -- mirror-admin(a)lists.fedoraproject.org
>  To unsubscribe send an email to mirror-admin-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]...
>  Do not reply to spam, report it:
>  https://pagure.io/fedora-infrastructure/new_issue
>  

-- 
Stephen J Smoogen.
Let us be kind to one another, for most of us are fighting a hard battle.
-- Ian MacClaren

[Kees Bakker]

On 06-10-2022 14:43, Stephen John Smoogen wrote:
>  On Thu, 6 Oct 2022 at 04:48, Kees Bakker <keesb(a)ghs.com&gt;
> wrote:
>
>      Hi,
>
>      Sorry if this the wrong mailing list for this. Or should I create an issue on
> github?
>
>
>  It may help to back up a bit and explain what the setup you are trying for. 
Good
point. The main reason is that CentOS and Fedora both come with .repo
files that use metalink=
In my setup (disconnected from the Internet) I can't use that. As a workaround
I manually figured out what baseurl= settings I need. This is doable but quite
a hassle when creating containers all the time. It certainly is not possible with
a sed / bash script.
BTW. I do have a copy of a CentOS and Fedora mirror on my local network.
So, I was hoping to setup a MirrorManager and just have my own internal
mirror on it.

>  Mirror manager is meant to deal with dozens of geolocated mirrors on
> the internet and work out which ones are closest to the user and give a list of some
> subset to them. It is also meant to allow for self-management of each of the mirrors and
> then scanning of the sites to see if they are in sync.
>
>  If you aren't needing all of this, then mirrorlist may be overkill for what you need.
> Air gapped networks generally do not have a way to give geolocation to the site (as that
> would usually be a bad thing on an air-gapped network). They are also usually more
> centrally managed.
>
>  The two sites I know which are using mirrormanager are Fedora and RPMfusion. I expect
> that it has been set up in situ for a long time so having it looked at by someone outside
> to figure out what isn't documented is good. 
So, I draw the conclusion that
MirrorManager is not going to give what I was hoping for.

Thanks for the answer anyway.

>      I'm trying to setup my own MirrorManager [1]. The README gives two simple
> methods,
>      but none of them work for me. The Vagrant method only works when there is an IPA
>      server (ipa.tinystage.test) around. The other method (native, virtualenv) has lots
> of
>      issues. The setup description is close to useless.
>
>      Problems encountered so far:
>      * no example or description for client_secrets.json
>      * pip requirements needs itsdangerous==2.0.1
>      * flaskext is not ready for Python3
>         File
> "/home/keesb/src/mirrormanager2/.env/lib64/python3.10/site-packages/flaskext/xmlrpc.py",
> line 252
>           except Fault, fault:
>                  ^^^^^^^^^^^^
>      SyntaxError: multiple exception types must be parenthesized
>
>
>      So, the question is: how can I setup my own MirrorManager? Did anyone try this
>      recently?
>
>      [1] https://github.com/fedora-infra/mirrormanager2/
>      -- 
>      Kees
>      _______________________________________________
>      Mirror-admin mailing list -- mirror-admin(a)lists.fedoraproject.org
>      To unsubscribe send an email to mirror-admin-leave(a)lists.fedoraproject.org
>      Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>      List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>      List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]...
>      Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
>
>
>
>  -- 
>  Stephen J Smoogen.
>  Let us be kind to one another, for most of us are fighting a hard battle. -- Ian
> MacClaren

[Stephen John Smoogen]

On Thu, 6 Oct 2022 at 10:24, Kees Bakker <keesb(a)ghs.com&gt; wrote:

>  On 06-10-2022 14:43, Stephen John Smoogen wrote:
>
>  On Thu, 6 Oct 2022 at 04:48, Kees Bakker <keesb(a)ghs.com&gt; wrote:
>
> > Hi,
> >
> > Sorry if this the wrong mailing list for this. Or should I create an
> > issue on github?
> >
> >
>  It may help to back up a bit and explain what the setup you are trying for.
>
>  Good point. The main reason is that CentOS and Fedora both come with .repo
>  files that use metalink=
>  In my setup (disconnected from the Internet) I can't use that. As a
>  workaround
>  I manually figured out what baseurl= settings I need. This is doable but
>  quite
>  a hassle when creating containers all the time. It certainly is not
>  possible with
>  a sed / bash script.
>  BTW. I do have a copy of a CentOS and Fedora mirror on my local network.
>  So, I was hoping to setup a MirrorManager and just have my own internal
>  mirror on it.
>
>  
Well I was going to say you might do better with mirrorlist= lines and
using the CentOS mirrorlist software but that would also need changes.
Normally I just do one of the following:
1. Copy out the repo files with one pointing to the one I want.
2. Have a replacement RPM which has the repo files i want which replaces
fedora-release/centos-release/rocky-release/etc.
3. Have an additional RPM with working repo files and use a %post to
sed/awk/etc the existing repos.
I looked for other mirrorlink scripts but they all look like they need more
work to make work than mirrormanager.

The main developer for mirrormanager is usually about a couple of days off
from reading the list so they may have answers also.


>  Mirror manager is meant to deal with dozens of geolocated mirrors on
> the
>  internet and work out which ones are closest to the user and give a list of
>  some subset to them. It is also meant to allow for self-management of each
>  of the mirrors and then scanning of the sites to see if they are in sync.
>
>  If you aren't needing all of this, then mirrorlist may be overkill for
>  what you need. Air gapped networks generally do not have a way to give
>  geolocation to the site (as that would usually be a bad thing on an
>  air-gapped network). They are also usually more centrally managed.
>
>  The two sites I know which are using mirrormanager are Fedora and
>  RPMfusion. I expect that it has been set up in situ for a long time so
>  having it looked at by someone outside to figure out what isn't documented
>  is good.
>
>  So, I draw the conclusion that MirrorManager is not going to give what I
>  was hoping for.
>
>  Thanks for the answer anyway.
>
>
>
>
> > I'm trying to setup my own MirrorManager [1]. The README gives two simple
> > methods,
> > but none of them work for me. The Vagrant method only works when there is
> > an IPA
> > server (ipa.tinystage.test) around. The other method (native, virtualenv)
> > has lots of
> > issues. The setup description is close to useless.
> >
> > Problems encountered so far:
> > * no example or description for client_secrets.json
> > * pip requirements needs itsdangerous==2.0.1
> > * flaskext is not ready for Python3
> >    File
> >
> "/home/keesb/src/mirrormanager2/.env/lib64/python3.10/site-packages/flaskext/xmlrpc.py",
> > line 252
> >      except Fault, fault:
> >             ^^^^^^^^^^^^
> > SyntaxError: multiple exception types must be parenthesized
> >
> >
> > So, the question is: how can I setup my own MirrorManager? Did anyone try
> > this
> > recently?
> >
> > [1] https://github.com/fedora-infra/mirrormanager2/
> > --
> > Kees
> > _______________________________________________
> > Mirror-admin mailing list -- mirror-admin(a)lists.fedoraproject.org
> > To unsubscribe send an email to
> > mirror-admin-leave(a)lists.fedoraproject.org
> > Fedora Code of Conduct:
> > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
> > https://lists.fedoraproject.org/archives/list/[e-mail]...
> > Do not reply to spam, report it:
> > https://pagure.io/fedora-infrastructure/new_issue
> >
>
>
>  --
>  Stephen J Smoogen.
>  Let us be kind to one another, for most of us are fighting a hard battle.
>  -- Ian MacClaren
>
>
>  
-- 
Stephen J Smoogen.
Let us be kind to one another, for most of us are fighting a hard battle.
-- Ian MacClaren