No more deltarpms by default

[Jonathan Dieter]

FWIW, I wrote and maintained yum-presto before it was integrated into 
yum.  I've commented inline:

On 10/06/2014 06:31 PM, Kevin Fenzi wrote:
> On Mon, 06 Oct 2014 09:07:33 -0400
> Stephen Gallagher <sgallagh at redhat.com> wrote:
>
>> The deltarpms were meant to serve two purposes
>>
>> 1) (lesser) Address the needs of users in developing countries (where
>> Fedora is fairly popular) and bandwidth concerns are very
>> considerable. Many of these users have connections that are either
>> metered or extremely slow, so deltarpms provides a way to get the
>> data to them more economically. This of course can be handled with a
>> non-default option, so we can talk about making that more
>> discoverable if we disable them by default.

When I wrote yum-presto it was for this reason.  We were on a lousy link 
and I couldn't run a local mirror without deltarpms.  Having said that, 
with download caps now being a thing in certain developed countries, 
this is no longer just a "developing countries" problem.

>> 2) (Major) Deltarpms significantly (Kevin, can you comment with
>> numbers?) reduces the load on the Fedora update servers and mirrors,
>> thereby reducing hosting costs as well as increasing the efficiency of
>> the available bandwidth (since smaller downloads mean you will be
>> tying up the line for a shorter amount of time).
>
> I don't recall this ever being a reason. (I could be wrong).
>
> I think this actually is worse for mirrors in some ways. They have to
> mirror a bunch more files and take up space (which is very dear to
> mirrors). It does get them less BW used, but most mirrors are on big
> links and don't really notice.
>
> It's definitely worse for releng. Generating drpms takes a long time
> and delays things like updates pushes or composes.

+1

>> It would be nice to see if we can find ways to improve the performance
>> of the deltarpm reconstruction instead. Much of the time is spent on
>> compression/decompression tasks which *should* be massively parallel;
>> we should be able to speed this up by taking advantage of additional
>> cores (and hyperthreading) on the system, for example. Another option
>> might be not to bother recompressing the RPMs but instead just
>> install an uncompressed RPM (and possibly recompress it out of band
>> if we needed to keep it in the cache).
>
> Folks wanting to do so are welcome to help out...
>
> Get to coding. ;)

As mentioned elsewhere, the problem *is* signatures.  yum (quite 
rightly) refuses to install an rpm whose signature doesn't match the one 
in the primary repodata.  And I believe that the signature in the RPM is 
also over the whole compressed rpm.  To make this work, we'd need to add 
an "uncompressed" signature for every package to the primary repodata as 
well as probably the rpms themselves.

We have already written the code to have yum run applydeltarpm in 
parallel according to the number of processors on a system, but it 
remains a single-threaded application that spends most of its time 
recompressing the data.

Finally, if we do want to stop using deltarpms by default, I think the 
easiest thing to do would be to set dnf to use deltarpms if deltarpm is 
not installed, remove the deltarpm requirement that dnf has, and remove 
deltarpm from default installs.  Then upgrades don't get unexpected 
changes in behavior and new installs can be told that getting deltarpm 
support is as easy as "dnf install deltarpm".

Jonathan