Metalink support

[Bram Neijt]

Yes it sounds like a nice idea and adding the chunk checksums is nice,
but I doubt anybody on this list would like those checksums to be
calculated on any of the infrastructure machines at fedora. And
although I could easily run a service that would generate metalinks
for you guys, I doubt anybody would like to incorporate it in the
official site.

Because the SHA1 is already known on the mirror, the updates of
metalinks can be done very quickly (a script would just do some text
parsing and posting) and it would allow it to easily create metalinks
and even keep them up to date (say weekly?).

Another option is to patch MM, but I think you should also include the
SHA1 so that would mean you would have to place file metadata in the
database. If you are going to do that, it might be easier to run
Bouncer and use the already existing patch for that.

Simply put: would a solution which runs outside of MM be acceptable?

Bram

On Mon, Mar 17, 2008 at 2:31 PM, Jeroen van Meeuwen <kanarip at kanarip.com> wrote:
> Bram Neijt wrote:
>  > So would I, but for what I've seen the SHA1 information is not part of
>  > the mirrormanager database.
>  >
>  > For any "good" implementation of metalinks in the mirrormanager, I
>  > would at least want trustworthy SHA1 information in the database
>  > otherwise the only advantage of using metalinks would be load
>  > balancing on your mirrors (which is already mostly fixed with MM).
>  >
>  > So for the end user, there is not much to gain if metalinks is
>  > implemented in the mirrormanager. The only solution for the SHA1
>  > information I could think of was to implement a script on the main
>  > mirror which hosts the SHA1 files, which would mean not running on the
>  > server with the MM database, which would mean not having direct access
>  > to that database.
>  >
>  > Parsing HTML is pretty stupid, I agree, but there is probably a much
>  > cleaner way of getting the mirror list (I havn't worked with turbo
>  > gears before, so I couldn't find a view of the list I would want).
>  > Something like fedora-test-data/mirror-hosts-core.txt would be much
>  > better.
>  >
>  > Rene is still working on the MM patch, but without the SHA1
>  > information, I'm not really sure it would be _my_ ideal solution.
>  >
>  > Greets,
>  >    Bram
>  >
>
>  How about using:
>
>  http://mirrors.fedoraproject.org/mirrorlist?path=pub/fedora/linux/releases/8/Fedora/i386/iso/Fedora-8-i386-DVD.iso
>
>  instead of the HTML?
>
>  Incorporating that in the .metalink, trusting Mirrormanager with
>  managing whether that file is actually the file that is supposed to be
>  there, and the metalink client doing a check-up on the chunks it has
>  downloads and see if they match? Creating a metalink would then consist of:
>
>  - getting the iso
>  - verifying the sha1sum (so you don't start off on the wrong foot)
>  - creating chunks and their sha1sum for use in a .metalink file
>  - while/before the .metalink is being downloaded by a client,
>  incorporate the mirrormanager results and trust mirrormanager to return
>  the 1) right mirrors, 2) the right files, 3) the files with correct content.
>  - optionally allow the .metalink to be created on-the-fly with all
>  parameters mirrormanager has (&country=NL,DE,BE, but not &redirect=1)
>
>  Does that sound like a good idea?
>
>  Just my brain dump ;p
>
>  Kind regards,
>
>  Jeroen van Meeuwen
>  -kanarip
>
>
>
>  _______________________________________________
>  Fedora-infrastructure-list mailing list
>  Fedora-infrastructure-list at redhat.com
>  https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
>