Thoughts and question about MM2's UMDL script

[Adrian Reber]

On Fri, Jun 26, 2015 at 11:04:35AM -0600, Stephen John Smoogen wrote:
> On 26 June 2015 at 10:11, Pierre-Yves Chibon <pingou at pingoured.fr> wrote:
> > So the UMDL script currently supports three ways of crawling the tree:
> > * file
> > * rsync
> > * directory
> >
> > We, in Fedora, are only using the last one. I believe the `rsync` mode was added
> > to support Ubuntu and the file mode is basically a simplified version of the
> > directory mode, but that we do not use at at the moment.
> >
> > I would like to propose that we drop support for rsync. I feel that it may be
> > simpler and easier to create an UMDL and a crawler for each distro that would
> > like to use MirrorManager than maintaining a one-script-fits-all UMDL that is
> > in fact tested for only one of the scenario.
> > That being said, if we ever have interest from Ubuntu, CentOS or any other
> > communities, we should definitively look into making the UMDL and crawler as
> > re-usable as possible for them, but keeping the distro-specific bits separated.
> >
> 
> I thought rsync was added for speed versus Ubuntu and that we actually
> use it a lot on the crawlers... because that is the reason we needed
> to move the memory usage up.. it uses a lot of memory but takes a lot
> shorter period to 'crawl' or am I conflating two different things?

That's the rsync of the crawler scanning the mirrors, that's much faster
compared to http/ftp. umdl analyzing the master mirror can either use
the local filesystem or rsync or a file as input.

		Adrian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 811 bytes
Desc: not available
URL: <http://lists.fedoraproject.org/pipermail/infrastructure/attachments/20150626/39abe396/attachment.sig>