Getting fedora-enchilada content - quick-fedora-mirror never finishes?

[Russell Jones]

Hi all,

I run an established public mirror for EPEL and Rocky, and wanted to go
ahead and add Fedora into it as well.

I am running the quick-fedora-script for this. Started it around 10am
yesterday, checked on it today around 8am and it still had not even began
downloading anything yet. Just spinning the CPU core it's using at 100%.
The log file showed it still at "Receiving file list" stage.

22 hours, no progress. That's insane.

Is there a better supported way of getting the initial sync? Or is it
really just wait it out and hope it finishes?

[Rafal Maszkowski]

Dnia Fri, May 19, 2023 at 08:27:39AM -0500, Russell Jones napisał(a):
>  I am running the quick-fedora-script for this. Started it around
> 10am
>  yesterday, checked on it today around 8am and it still had not even began
>  downloading anything yet. Just spinning the CPU core it's using at 100%.
>  The log file showed it still at "Receiving file list" stage.
>  22 hours, no progress. That's insane. 
The script is not quite easy to run so I rather expect some
configuration mistake, not such a slowness.

>  Is there a better supported way of getting the initial sync? Or is
> it
>  really just wait it out and hope it finishes? 
Just rsync it from elsewhere, e.g. from the source you use in the script.

R.
-- 
„Walczy on z całym zapamiętaniem przeciwko intelektowi” - z akt personalnych prof. A.
Baeumlera

[Russell Jones]

Thanks,

I considered trying just a bare rsync, but that's where it's stuck at is
the rsync process.

I'll try just rsync with the flags I use for mirroring EPEL (I don't use
the quick sync script for that repo), see if it makes a difference.

On Fri, May 19, 2023 at 8:41 AM Rafal Maszkowski <rzm(a)icm.edu.pl&gt; wrote:

>  Dnia Fri, May 19, 2023 at 08:27:39AM -0500, Russell Jones
> napisał(a):
>  > I am running the quick-fedora-script for this. Started it around 10am
>  > yesterday, checked on it today around 8am and it still had not even began
>  > downloading anything yet. Just spinning the CPU core it's using at 100%.
>  > The log file showed it still at "Receiving file list" stage.
>  > 22 hours, no progress. That's insane.
>
>  The script is not quite easy to run so I rather expect some
>  configuration mistake, not such a slowness.
>
>  > Is there a better supported way of getting the initial sync? Or is it
>  > really just wait it out and hope it finishes?
>
>  Just rsync it from elsewhere, e.g. from the source you use in the script.
>
>  R.
>  --
>  „Walczy on z całym zapamiętaniem przeciwko intelektowi” - z akt
>  personalnych prof. A. Baeumlera
>

[Russell Jones]

OK, yeah, it's already actually grabbing content now. Took all of 10
seconds.

Something's up with that quick sync script. I'll stick to normal rsync.

On Fri, May 19, 2023 at 8:48 AM Russell Jones <arjones85(a)gmail.com&gt; wrote:

>  Thanks,
>
>  I considered trying just a bare rsync, but that's where it's stuck at is
>  the rsync process.
>
>  I'll try just rsync with the flags I use for mirroring EPEL (I don't use
>  the quick sync script for that repo), see if it makes a difference.
>
>  On Fri, May 19, 2023 at 8:41 AM Rafal Maszkowski <rzm(a)icm.edu.pl&gt; wrote:
>
> > Dnia Fri, May 19, 2023 at 08:27:39AM -0500, Russell Jones napisał(a):
> > > I am running the quick-fedora-script for this. Started it around 10am
> > > yesterday, checked on it today around 8am and it still had not even
> > began
> > > downloading anything yet. Just spinning the CPU core it's using at 100%.
> > > The log file showed it still at "Receiving file list" stage.
> > > 22 hours, no progress. That's insane.
> >
> > The script is not quite easy to run so I rather expect some
> > configuration mistake, not such a slowness.
> >
> > > Is there a better supported way of getting the initial sync? Or is it
> > > really just wait it out and hope it finishes?
> >
> > Just rsync it from elsewhere, e.g. from the source you use in the script.
> >
> > R.
> > --
> > „Walczy on z całym zapamiętaniem przeciwko intelektowi” - z akt
> > personalnych prof. A. Baeumlera
> >
>

[Rafal Maszkowski]

Dnia Fri, May 19, 2023 at 08:49:56AM -0500, Russell Jones napisał(a):
>  OK, yeah, it's already actually grabbing content now. Took all of
> 10
>  seconds.
>  Something's up with that quick sync script. I'll stick to normal rsync.
>  On Fri, May 19, 2023 at 8:48 AM Russell Jones <arjones85(a)gmail.com&gt; wrote:
>  > I considered trying just a bare rsync, but that's where it's stuck at is
>  > the rsync process.
>  > I'll try just rsync with the flags I use for mirroring EPEL (I don't use
>  > the quick sync script for that repo), see if it makes a difference. 
Maybe it was problem with the rsync? Once I managed to configure the
script it works for me quite well. It saves a lot of resouces and time.
It fails when partition is filled, and in that case I recover by a plain
rsync --delete ….

R.
-- 
„Walczy on z całym zapamiętaniem przeciwko intelektowi” - z akt personalnych prof. A.
Baeumlera

[Kevin Fenzi]

On Fri, May 19, 2023 at 03:56:36PM +0200, Rafal Maszkowski wrote:
>  Dnia Fri, May 19, 2023 at 08:49:56AM -0500, Russell Jones
> napisał(a):
>  > OK, yeah, it's already actually grabbing content now. Took all of 10
>  > seconds.
>  > Something's up with that quick sync script. I'll stick to normal rsync.
>  > On Fri, May 19, 2023 at 8:48 AM Russell Jones <arjones85(a)gmail.com&gt; wrote:
>  > > I considered trying just a bare rsync, but that's where it's stuck at
> is
>  > > the rsync process.
>  > > I'll try just rsync with the flags I use for mirroring EPEL (I don't
> use
>  > > the quick sync script for that repo), see if it makes a difference.
>  
>  Maybe it was problem with the rsync? Once I managed to configure the
>  script it works for me quite well. It saves a lot of resouces and time.
>  It fails when partition is filled, and in that case I recover by a plain
>  rsync --delete …. 
It really does save a lot of useless i/o on the master mirrors. ;)

That said, I ran into what sounds like the exact same issue here
recently. I wasn't fully able to pin it down, but I think it might be a
strange rsync bug. If the list of files to transfer is really large,
things go slower and slower over time and it never gets to transfering.

What version of rsync are you using there?
On the machine where I was seeing it, it was rhel9 with rsync-3.2.3-18.el9.x86_64
Updating it to the rsync version from fedora and it started working
normally again, but I also did a full rsync, make sure time was correct,
and a bunch of other things.

kevin

[Russell Jones]

This is 3.1.3 on Rocky 8. Yes as I watched the strace output, it'd start
out checking files pretty quick. When I checked it this morning it was
checking maybe 1 file a second.

I'll try compiling the latest version of rsync after I get this initial
sync done, and then running the quick sync script against my
already-downloaded files with the new version of rsync. See if it makes a
difference.



On Fri, May 19, 2023 at 11:35 AM Kevin Fenzi <kevin(a)scrye.com&gt; wrote:

>  On Fri, May 19, 2023 at 03:56:36PM +0200, Rafal Maszkowski wrote:
>  > Dnia Fri, May 19, 2023 at 08:49:56AM -0500, Russell Jones napisał(a):
>  > > OK, yeah, it's already actually grabbing content now. Took all of 10
>  > > seconds.
>  > > Something's up with that quick sync script. I'll stick to normal
> rsync.
>  > > On Fri, May 19, 2023 at 8:48 AM Russell Jones <arjones85(a)gmail.com&gt;
>  wrote:
>  > > > I considered trying just a bare rsync, but that's where it's
> stuck
>  at is
>  > > > the rsync process.
>  > > > I'll try just rsync with the flags I use for mirroring EPEL (I
> don't
>  use
>  > > > the quick sync script for that repo), see if it makes a difference.
>  >
>  > Maybe it was problem with the rsync? Once I managed to configure the
>  > script it works for me quite well. It saves a lot of resouces and time.
>  > It fails when partition is filled, and in that case I recover by a plain
>  > rsync --delete ….
>
>  It really does save a lot of useless i/o on the master mirrors. ;)
>
>  That said, I ran into what sounds like the exact same issue here
>  recently. I wasn't fully able to pin it down, but I think it might be a
>  strange rsync bug. If the list of files to transfer is really large,
>  things go slower and slower over time and it never gets to transfering.
>
>  What version of rsync are you using there?
>  On the machine where I was seeing it, it was rhel9 with
>  rsync-3.2.3-18.el9.x86_64
>  Updating it to the rsync version from fedora and it started working
>  normally again, but I also did a full rsync, make sure time was correct,
>  and a bunch of other things.
>
>  kevin
>  _______________________________________________
>  Mirror-admin mailing list -- mirror-admin(a)lists.fedoraproject.org
>  To unsubscribe send an email to mirror-admin-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]...
>  Do not reply to spam, report it:
>  https://pagure.io/fedora-infrastructure/new_issue
>