No more deltarpms by default

[Nico Kadel-Garcia]

On Sun, Oct 19, 2014 at 5:26 AM, Reindl Harald <h.reindl at thelounge.net> wrote:
>
> Am 19.10.2014 um 06:37 schrieb Nico Kadel-Garcia:
>>
>> On Sat, Oct 18, 2014 at 8:28 PM, Reindl Harald <h.reindl at thelounge.net>
>> wrote:
>>>
>>> #!/usr/bin/bash
>>> basearch=`uname -i`
>>> releasever=`rpm -q --qf "%{version}\n" fedora-release`
>>> for g in `ls -1b /var/cache/yum`
>>> do
>>>   if [ -d /var/cache/yum/$g/packages ]
>>>   then
>>>    echo "/var/cache/yum/$g/packages/ > /repo/cache/fc$releasever/"
>>>    sudo mv --verbose /var/cache/yum/$g/packages/*.rpm
>>> /repo/cache/fc$releasever/ 2> /dev/null
>>>   fi
>>> done
>>> /buildserver/repo-create.sh
>>
>>
>> And that only works on one machine, and is sensitive to exclusions
>> written into yum.conf or /etc/yum.repos.d/*, etc., etc., etc. You also
>> forgot to use 'gpgcheck' to verify the contents of the already
>> downloaded RPM, since 'repsync' does not track successful versus
>> partial downloads without it.
>>
>> Anyway. Now multiply your 'reposync' based tool by 200 thin
>> provisioned virtual machines, and you're chewing up quite a lot of
>> disk space doing this
>
>
> you don't get it
>
> * that single machine has all used packages installed
> * that single machine builds a repo below /repo/cache/fc$releasever/
> * that repo don't mirror blindly anything, just used RPM's
> * that RPMs are the result *after* apply deltarpm out of yum cache

I get it. It starts with the the fact that your script has not
verified *any* of your RPM's, and reposync does not ordinarily verify
file contents unless you activate GPG verification. So any partial
transfer or interrupted transfer will effectively corrupted your local
repo, especially if anything else is dependent on that package. Do
take a look at the github scripts I pointed out, they do a more
thorough job.

And running 'reposync" still downloads the upstream repodata. You've
really saved very little bandwidth as long as you keep doing this, and
you *need* to keep downloading the bulky repodata to ensure getting
the latest releases of packages. Yum repodata itself has no 'deltarps'
for its own information, it's compressed binary database blobs.

Last, doing this on an individual machine does not scale You're
chewing up disk space on each machine, and bandwitdh, with all the
required 'repodata' downloads.