Hi everyone,
I'm orphaning deltarpm because, as it's currently used in Fedora, it's not very effective, bugs keep getting opened against it because it's not working as well as it should (mostly an infra issue as opposed to a problem with the tool itself), and I no longer have the time or motivation to deal with these tickets when it seems like there's not much benefit.
The main problem with deltarpms is that we don't keep them from compose to compose, so we're only getting the previous day's deltarpms when we download updates. This means that the only way you'll be able to truly take advantage of deltarpms is if you get updates after every compose.
For more details, see: https://pagure.io/releng/issue/7215
Bugs that need to be looked into:
https://bugzilla.redhat.com/show_bug.cgi?id=1873876 - This is mostly two different problems: 1) when the zstd algorithm changes, deltarpms cannot be rebuilt to the exact same RPM, and 2) deltarpms for the kernel package seem to be impossible to rebuild to the exact same RPM due to the compression format used on the kernel RPMs
https://bugzilla.redhat.com/show_bug.cgi?id=2058477 - No idea why 100+ MB of packages failed to build correctly (no logs in the ticket), but it's probably related to (1) above.
A brief history of how deltarpms came to be added to Fedora: https://www.jdieter.net/posts/2013/05/31/eulogy-for-yum-presto/
Finally, I want to say a huge thank you to Michael Schroeder, who wrote deltarpm. The tool itself still works beautifully well, and the issues we're seeing are all infrastructure related. I also want to say thank you to everyone who worked on getting deltarpms into Fedora in the first place and who have helped keep them running this long.
Jonathan
On Sun, Mar 6, 2022 at 8:51 AM Jonathan Dieter jdieter@gmail.com wrote:
Hi everyone,
I'm orphaning deltarpm because, as it's currently used in Fedora, it's not very effective, bugs keep getting opened against it because it's not working as well as it should (mostly an infra issue as opposed to a problem with the tool itself), and I no longer have the time or motivation to deal with these tickets when it seems like there's not much benefit.
I have always disliked deltarpms. They expand the size of the frequently downloaded metadata with little overall benefit.
On 3/6/22 13:29, Nico Kadel-Garcia wrote:
On Sun, Mar 6, 2022 at 8:51 AM Jonathan Dieter jdieter@gmail.com wrote:
Hi everyone,
I'm orphaning deltarpm because, as it's currently used in Fedora, it's not very effective, bugs keep getting opened against it because it's not working as well as it should (mostly an infra issue as opposed to a problem with the tool itself), and I no longer have the time or motivation to deal with these tickets when it seems like there's not much benefit.
I have always disliked deltarpms. They expand the size of the frequently downloaded metadata with little overall benefit.
I have also strongly disliked deltarpms. They very rarely help and significantly increase attack surface.
On 3/6/22 13:39, Demi Marie Obenour wrote:
I have also strongly disliked deltarpms. They very rarely help and significantly increase attack surface.
If deltarpm succeeds and both the old .rpm and the new.rpm are signed, then how is the attack surface larger, as long as any consumer verifies the signature?
On 3/6/22 18:10, John Reiser wrote:
I have also strongly disliked deltarpms. They very rarely help and significantly increase attack surface.
If deltarpm succeeds and both the old .rpm and the new.rpm are signed, then how is the attack surface larger, as long as any consumer verifies the signature?
This assumes that deltarpm (the program) does not contain any security flaws of its own, which could allow for code execution while the deltarpm is being applied. This is a bad assumption: a cursory audit I did found that it is not designed with untrusted input in mind. The code is also quite hard to follow, which makes auditing it quite difficult. Finally, it exposes decompression libraries to untrusted input before signature verification, and it itself has at the very least several areas where a bad deltarpm could cause it to allocate gigabytes of RAM.
On 3/6/22 10:29, Nico Kadel-Garcia wrote:
I have always disliked deltarpms. They expand the size of the frequently downloaded metadata with little overall benefit.
Wasn't Jonathan's point that the benefit is "little" because of infrastructure issues?
I remember in the early days of deltarpm, it would frequently reduce the download size on my systems by 70-90%. I know that some people disliked that it made updates slower, but I always thought that reducing the bandwidth costs at our mirrors was a laudable goal. Good stewardship of resources, and all of that.
On Mon, Mar 7, 2022 at 5:03 AM Gordon Messmer gordon.messmer@gmail.com wrote:
I remember in the early days of deltarpm, it would frequently reduce the download size on my systems by 70-90%. I know that some people disliked that it made updates slower, but I always thought that reducing the bandwidth costs at our mirrors was a laudable goal. Good stewardship of resources, and all of that.
As with all else, "Your mileage WILL vary". In my experience (in a 1st world bandwidth world), the overheads of rebuilding the rpm (using the delta rpm base on a slow (SD card) disk on a slow (arm) system) far far far exceeded the reduced download times.
One of the first things I did on a new install was remove deltarpm, and add it to the global exclude list so it would not get installed on a system/package upgrade. It was a pure win for my systems. Obviously your experience will vary.
Dne 07. 03. 22 v 6:29 Gary Buhrmaster napsal(a):
One of the first things I did on a new install was remove deltarpm, and add it to the global exclude list so it would not get installed on a system/package upgrade. It was a pure win for my systems. Obviously your experience will vary.
Why so complicated?
Just setting `deltarpm` to False in dnf.conf do the same. Just saying.
Miroslav
Just setting `deltarpm` to False in dnf.conf do the same. Just saying.
The deltarpm option has low discoverability. The information is available, but the discovery chain is too long and not explicit enough. The main configuration file /etc/dnf/dnf.conf does not list each option, its default value, and perhaps a short comment. The main configuration file does not reference "# See 'man 5 dnf.conf'."
On 3/14/22 09:34, John Reiser wrote:
Just setting `deltarpm` to False in dnf.conf do the same. Just saying.
The deltarpm option has low discoverability. The information is available, but the discovery chain is too long and not explicit enough. The main configuration file /etc/dnf/dnf.conf does not list each option, its default value, and perhaps a short comment. The main configuration file does not reference "# See 'man 5 dnf.conf'."
Why wouldn't that be the first thing you tried anyway?
On 3/14/22 12:23, Samuel Sieb wrote:
On 3/14/22 09:34, John Reiser wrote:
Just setting `deltarpm` to False in dnf.conf do the same. Just saying.
The deltarpm option has low discoverability. The information is available, but the discovery chain is too long and not explicit enough. The main configuration file /etc/dnf/dnf.conf does not list each option, its default value, and perhaps a short comment. The main configuration file does not reference "# See 'man 5 dnf.conf'."
Why wouldn't that be the first thing you tried anyway?
The first thing is to look at the manual page "man dnf", whose FILES section points to the main configuration file /etc/dnf/dnf.conf. I expect dnf.conf to list all the options and their default values, or if not, then to point to the documentation which does. The manual page "man dnf" says nothing about deltarpm. The default /etc/dnf/dnf.conf lists only a few options with no individual documentation, and contains no explicit pointer to more documentation. I expect better: all reasonable attempts should lead quickly to the relevant documentation. This is the meaning of "good discoverability". If I get anywhere close, then I should find the answer or a direct pointer to the answer. Enable multiple pathways (both the main manual page AND the configuration tile should point to "man 5 dnf.conf") so as to reduce the number of steps.
On 3/14/22 15:23, John Reiser wrote:
On 3/14/22 12:23, Samuel Sieb wrote:
On 3/14/22 09:34, John Reiser wrote:
Just setting `deltarpm` to False in dnf.conf do the same. Just saying.
The deltarpm option has low discoverability. The information is available, but the discovery chain is too long and not explicit enough. The main configuration file /etc/dnf/dnf.conf does not list each option, its default value, and perhaps a short comment. The main configuration file does not reference "# See 'man 5 dnf.conf'."
Why wouldn't that be the first thing you tried anyway?
The first thing is to look at the manual page "man dnf", whose FILES section points to the main configuration file /etc/dnf/dnf.conf.
Apparently you missed the "see also" section in the dnf man page: SEE ALSO • dnf.conf(5), DNF Configuration Reference
I expect dnf.conf to list all the options and their default values, or if not, then to point to the documentation which does. The manual page "man dnf" says nothing about deltarpm. The default /etc/dnf/dnf.conf lists only a few options with no individual documentation, and contains no explicit pointer to more documentation. I expect better: all reasonable attempts should lead quickly to the relevant documentation. This is the meaning of "good discoverability". If I get anywhere close, then I should find the answer or a direct pointer to the answer. Enable multiple pathways (both the main manual page AND the configuration tile should point to "man 5 dnf.conf") so as to reduce the number of steps.
Having the default values listed in the config file can be nice, but it can also make it difficult to find changes. I certainly don't expect the config file to point me at the man page. That seems rather pointless. I do expect the man page to exist though.
Dne 14. 03. 22 v 23:23 John Reiser napsal(a):
The default /etc/dnf/dnf.conf lists only a few options with no individual documentation, and contains no explicit pointer to more documentation. I expect better: all reasonable attempts should lead quickly to the relevant documentation. This is the meaning of "good discoverability". If I get anywhere close, then I should find the answer or a direct pointer to the answer. Enable multiple pathways (both the main manual page AND the configuration tile should point to "man 5 dnf.conf") so as to reduce the number of steps.
Make a wish....
https://github.com/rpm-software-management/dnf/pull/1820
Miroslav