Hi all,
I think deltarpm is not really useful anymore: - there are very few drpm files in the repository, see for example: https://download.fedoraproject.org/pub/fedora/linux/updates/34/Everything/x8... https://download.fedoraproject.org/pub/fedora/linux/updates/33/Everything/x8... - those that actually are there, are mostly about small packages anyway - personally, I haven't seen it being used for a long time - there is also argument that people's connection bandwidth nowadays tends to be fast enough to make the package rebuilding actually slower than downloading the whole package (but that really vary between different installations) - and most importantly: drpm files are - by design - processed before checking the package signature, which exposes rather big attack surface(*)
Can deltarpm be disabled by default? In the few cases where it's actually useful (if there are any...), user is free to enable it, but the default would be significantly more secure this way.
(*) it is integrity protected via a hash in the repository metadata, but repository metadata in Fedora are still not signed - so this all heavily depends on the integrity of the [HTTPS connection to] mirrors.fedoraproject.org server (or any of CAs trusted by the system) - a rather fragile single point of failure.
On Wed, Aug 11, 2021 at 10:03:50PM +0200, Marek Marczykowski-Górecki wrote:
Hi all,
I think deltarpm is not really useful anymore:
- there are very few drpm files in the repository, see for example: https://download.fedoraproject.org/pub/fedora/linux/updates/34/Everything/x8... https://download.fedoraproject.org/pub/fedora/linux/updates/33/Everything/x8...
- those that actually are there, are mostly about small packages anyway
- personally, I haven't seen it being used for a long time
- there is also argument that people's connection bandwidth nowadays tends to be fast enough to make the package rebuilding actually slower than downloading the whole package (but that really vary between different installations)
Yeah. ;(
- and most importantly: drpm files are - by design - processed before checking the package signature, which exposes rather big attack surface(*)
Thats not the case.
Can deltarpm be disabled by default? In the few cases where it's actually useful (if there are any...), user is free to enable it, but the default would be significantly more secure this way.
I do think we should drop drpms or make them more useful, but I don't think there's any security angle here. (see below)
(*) it is integrity protected via a hash in the repository metadata, but repository metadata in Fedora are still not signed - so this all heavily depends on the integrity of the [HTTPS connection to] mirrors.fedoraproject.org server (or any of CAs trusted by the system) - a rather fragile single point of failure.
drpms work by downloading the delta, then using it + the version you have installed to recreate the signed rpm (just like you downloaded the full signed update) and then the gpg signature is checked of that full rpm, just like one you downloaded. If the drpm is tampered with it won't reassemble and it will fall back to the full signed rpm.
Additionally, fedoraproject.org has dnssec enabled, so if you are worried, do enable that to avoid hyjacking.
kevin
On Wed, Aug 11, 2021 at 02:02:39PM -0700, Kevin Fenzi wrote:
drpms work by downloading the delta, then using it + the version you have installed to recreate the signed rpm (just like you downloaded the full signed update)
I'm worried about this process specifically. It does rather heavy parsing on a drpm file, before having a chance to verify the signature.
and then the gpg signature is checked of that full rpm, just like one you downloaded. If the drpm is tampered with it won't reassemble and it will fall back to the full signed rpm.
Additionally, fedoraproject.org has dnssec enabled, so if you are worried, do enable that to avoid hyjacking.
Ok, so there are several things here: 1. DNSSEC protects against only some of the threats. It won't (on its own) help if someone intercepts my TCP connection when I download updates via wifi in a coffee shop.
2. HTTPS provides _much_ better protection against various kind of MitM attacks, but in some threat models is still not enough: - It protects only the connection to the server, not integrity of the mirrors.fedoraproject.org machine itself (if someone manages to exploit a hypothetical bug in the web server there, it's game over). - Furthermore, there is a wildcard cert for *.fedoraproject.org, so in practice it's enough to break into any of the servers (having key for that cert), to make MitM attack feasible. - There are over hundred trusted CAs in the default setup. It takes just one compromised/malicious to issue a duplicate cert. Both scenarios has happened to some CAs in the past.
3. Finally, this is about just Fedora repositories. It doesn't help for much for dozen other repositories that user is free to add. Some may even use plain HTTP (and rely just on package signatures)!
While I still think disabling deltarpm by default is a good idea, in absence of signed metadata there are a few things that could improve the situation: - Pin specific CA in the repo config (sslcacert option) - Add CAA record for the same purpose - Avoid wildcard cert for such critical purpose, but to make it really worth it, the wildcard cert for the domain would need to not exists - rather impractical thing. Maybe use separate domain then (getfedora.org?)? - Some would propose to use own CA to avoid trusting the whole DigiCert (or other single CA), but personally I think the downsides overweights the benefits
And this is just about the connection part, not about integrity of the server itself... BTW, I do hope that signing keys are stored somewhere else.
On Thu, Aug 12, 2021 at 12:00:16AM +0200, Marek Marczykowski-Górecki wrote:
On Wed, Aug 11, 2021 at 02:02:39PM -0700, Kevin Fenzi wrote:
drpms work by downloading the delta, then using it + the version you have installed to recreate the signed rpm (just like you downloaded the full signed update)
I'm worried about this process specifically. It does rather heavy parsing on a drpm file, before having a chance to verify the signature.
Sure, and it takes cpu and disk space vs download time.
My understanding is that it's pretty simple, but I don't know if anyone has done any kind of security review of it.
and then the gpg signature is checked of that full rpm, just like one you downloaded. If the drpm is tampered with it won't reassemble and it will fall back to the full signed rpm.
Additionally, fedoraproject.org has dnssec enabled, so if you are worried, do enable that to avoid hyjacking.
Ok, so there are several things here:
- DNSSEC protects against only some of the threats. It won't (on its own) help if someone intercepts my TCP connection when I download updates via wifi in a coffee shop.
Sure. You can shift the threats and make yourself unsafe. ;(
- HTTPS provides _much_ better protection against various kind of MitM attacks, but in some threat models is still not enough:
- It protects only the connection to the server, not integrity of the mirrors.fedoraproject.org machine itself (if someone manages to exploit a hypothetical bug in the web server there, it's game over).
Sure, but also: The machines that built the rpms, the machines that store them, the machines that store the source, the maintainers that upload the source, upstreams that create the source, etc
- Furthermore, there is a wildcard cert for *.fedoraproject.org, so in practice it's enough to break into any of the servers (having key for that cert), to make MitM attack feasible.
True, although the number of servers that have that wildcard is limited. We specifically don't store it on just any machines, it's only on our proxies and most trusted machines.
- There are over hundred trusted CAs in the default setup. It takes just one compromised/malicious to issue a duplicate cert. Both scenarios has happened to some CAs in the past.
Sure, but this is one of those "do it and it works, but everyone knows" sorts of attacks. Once you do that once you get untrusted...
Transparency logs help out with this at least from a ecosystem angle.
- Finally, this is about just Fedora repositories. It doesn't help for
much for dozen other repositories that user is free to add. Some may even use plain HTTP (and rely just on package signatures)!
Sure, or packages downloaded from pkgs.org or curl foo.bar | bash.
While I still think disabling deltarpm by default is a good idea, in absence of signed metadata there are a few things that could improve the situation:
- Pin specific CA in the repo config (sslcacert option)
This would be limiting in the event we moved to another registrar...
- Add CAA record for the same purpose
This is I think doable from an infrastructure point of view. Of course dnf/librepo would need to know to check it.
- Avoid wildcard cert for such critical purpose, but to make it really worth it, the wildcard cert for the domain would need to not exists - rather impractical thing. Maybe use separate domain then (getfedora.org?)?
I don't think this is worth it.
- Some would propose to use own CA to avoid trusting the whole DigiCert (or other single CA), but personally I think the downsides overweights the benefits
Yeah, thats been proposed before. Pinning the cert and having a backup version thats self-signed. I don't think thats worth it either.
And this is just about the connection part, not about integrity of the server itself... BTW, I do hope that signing keys are stored somewhere else.
We use sigul for signing. So, the "keys" are on a sign vault machine (bare metal) that isn't running any listening services (it reaches out to the sign bridge). Also, it can't do anything with the keys itself, it needs at least one signer with their passphrase + the vaults passphrase to be able to decrypt the passphrase to sign things. Its pretty reasonable for a all software setup. https://pagure.io/sigul
kevin
On Wed, Aug 11, 2021 at 11:03 PM Kevin Fenzi kevin@scrye.com wrote:
On Wed, Aug 11, 2021 at 10:03:50PM +0200, Marek Marczykowski-Górecki wrote:
Hi all,
I think deltarpm is not really useful anymore:
- there are very few drpm files in the repository, see for example: https://download.fedoraproject.org/pub/fedora/linux/updates/34/Everything/x8... https://download.fedoraproject.org/pub/fedora/linux/updates/33/Everything/x8...
- those that actually are there, are mostly about small packages anyway
- personally, I haven't seen it being used for a long time
- there is also argument that people's connection bandwidth nowadays tends to be fast enough to make the package rebuilding actually slower than downloading the whole package (but that really vary between different installations)
Yeah. ;(
I actually wanted to bring up this topic a while ago, but then got side-tracked. In my experience, drpms have negligible effect on package downloads for updates.
To add some anecdata, these are the stats from today's "dnf distro-sync" transaction on my laptop, with a few days' worth of updates+updates-testing changes:
Install 4 Packages Upgrade 161 Packages Remove 3 Packages
Total download size: 751 M
[DRPM 1/3] python3-tqdm-4.61.1-1.fc34_4.62.0-1.fc34.noarch.drpm: done [DRPM 2/3] perl-Module-CoreList-5.20210620-1.fc34_5.20210723-1.fc34.noarch.drpm: done [DRPM 3/3] binutils-2.35.1-41.fc34_2.35.2-4.fc34.x86_64.drpm: done
Delta RPMs reduced 750.9 MB of updates to 747.5 MB (0.4% saved)
This about matches what I've been seeing on this laptop and my main machine for many months. Either there's 0-2% savings at most, or a few % of additional data downloads due to failed downloads or failed rpm reconstructions.
I wonder why there's so few drpms in most transactions I see? Does this system not prioritize packages, like, those that are installed on all variants, or installed by default on Workstation?
The way it is right now, I could turn drpms off entirely, and probably not change the download size at all (because savings are small and are cancelled out by failure cases), and save some CPU time. Is it really worth it keeping all that infrastructure for drpms around, if they doesn't actually provide any benefit wrt. amount of data to download, and actually increases CPU usage?
Fabio
On Thu, Aug 12, 2021 at 01:00:26AM +0200, Fabio Valentini wrote: ...snip...
I wonder why there's so few drpms in most transactions I see? Does this system not prioritize packages, like, those that are installed on all variants, or installed by default on Workstation?
So, it's complicated, but let me try...
drpms are generated by createrepo_c at the same time it's generating the repodata for a repo. You can pass it things to tell it how many deltas to make and where to find old rpms, etc.
When an updates push happens, bodhi fires off a pungi job for that version/repo (ie, say fedora-34-updates). pungi sees that as a new compose. It gathers all the things from koji in f34-updates tag and sets things up. When it runs createrepo_c it only has access to: the current rpms, the rpms in the current 'updates/34' repo and thats it. So, the only drpms we currently get are when there's a update already in stable updates and we have a new update of the same package going stable. But then it also only keeps it for that one updates push. On the next push things start over and there's not any 'old' package in the updates repo, so no drpm is made.
Lots of things make this not very flexable: * only can make drpms at createrepo_c run time, this means you can't do them later/on the side/keep previous ones around/etc. * pungi doesn't have access to older rpms at the right time to generate more and has no way to keep them accross pushes.
We could do lots of things to fix this, but would need work: * if we had a standlone tool that could generate arbitrary drpms and 'insert' them into repodata we could run those seperate from updates composes and keep more of them. Of course repodata signing would complicate this. * If dnf could look for drpms in some standard 'other' repo when enabled we could perhaps just generate those repos out of band.
It's also not clear what would be the ideal amount/kind of drpms to make... GA to current updates? That plus last update? That plus last 2 updates?
The way it is right now, I could turn drpms off entirely, and probably not change the download size at all (because savings are small and are cancelled out by failure cases), and save some CPU time. Is it really worth it keeping all that infrastructure for drpms around, if they doesn't actually provide any benefit wrt. amount of data to download, and actually increases CPU usage?
I recall when we made them enabled by default. I heard a number of users say that that specifically was why they decided to run Fedora. ;) Of course times may well have changed.
kevin
On Wed, Aug 11, 2021 at 10:03:50PM +0200, Marek Marczykowski-Górecki wrote: I do think we should drop drpms or make them more useful, but I don't think there's any security angle here. (see below)
drpms work by downloading the delta, then using it + the version you have installed to recreate the signed rpm (just like you downloaded the full signed update) and then the gpg signature is checked of that full rpm, just like one you downloaded. If the drpm is tampered with it won't reassemble and it will fall back to the full signed rpm.
Sorry to resurrect this thread.
Another issue - which is not per-se a security issue but it's still a problem - is that deltarpm uses md5 checksums pervasively. They're everywhere. And it uses its own implementation of md5 which doesn't respect FIPS, so even when the user has *explicitly* configured their system to not use md5 for anything security-relevant, libdeltarpm won't know or care.
On Sat, Nov 6, 2021 at 3:43 AM Daniel Alley dalley@redhat.com wrote:
On Wed, Aug 11, 2021 at 10:03:50PM +0200, Marek Marczykowski-Górecki wrote: I do think we should drop drpms or make them more useful, but I don't think there's any security angle here. (see below)
drpms work by downloading the delta, then using it + the version you have installed to recreate the signed rpm (just like you downloaded the full signed update) and then the gpg signature is checked of that full rpm, just like one you downloaded. If the drpm is tampered with it won't reassemble and it will fall back to the full signed rpm.
Sorry to resurrect this thread.
Another issue - which is not per-se a security issue but it's still a problem - is that deltarpm uses md5 checksums pervasively. They're everywhere. And it uses its own implementation of md5 which doesn't respect FIPS, so even when the user has *explicitly* configured their system to not use md5 for anything security-relevant, libdeltarpm won't know or care.
This is not true with libdrpm though, and that version is what createrepo_c uses.
That said, it *is* also possible to create DeltaRPMs separately from using createrepo_c, especially by building upon libdrpm, we just haven't done it.
That's a fair point, I was actually not aware that https://github.com/rpm-software-management/drpm contained a completely separate implementation of applydeltarpm.
On 11/6/21 6:47 AM, Neal Gompa wrote:
On Sat, Nov 6, 2021 at 3:43 AM Daniel Alley dalley@redhat.com wrote:
On Wed, Aug 11, 2021 at 10:03:50PM +0200, Marek Marczykowski-Górecki wrote: I do think we should drop drpms or make them more useful, but I don't think there's any security angle here. (see below)
drpms work by downloading the delta, then using it + the version you have installed to recreate the signed rpm (just like you downloaded the full signed update) and then the gpg signature is checked of that full rpm, just like one you downloaded. If the drpm is tampered with it won't reassemble and it will fall back to the full signed rpm.
Sorry to resurrect this thread.
Another issue - which is not per-se a security issue but it's still a problem - is that deltarpm uses md5 checksums pervasively. They're everywhere. And it uses its own implementation of md5 which doesn't respect FIPS, so even when the user has *explicitly* configured their system to not use md5 for anything security-relevant, libdeltarpm won't know or care.
This is not true with libdrpm though, and that version is what createrepo_c uses.
Yes, but createrepo_c isn’t what runs on end-user devices.
Sincerely,
Demi Marie Obenour (she/her/hers)
On Sat, Nov 06, 2021 at 07:43:02AM -0000, Daniel Alley wrote:
Another issue - which is not per-se a security issue but it's still a problem - is that deltarpm uses md5 checksums pervasively. They're everywhere. And it uses its own implementation of md5 which doesn't respect FIPS, so even when the user has *explicitly* configured their system to not use md5 for anything security-relevant, libdeltarpm won't know or care.
They are used as a consistency check, it might as well use crc32. So I don't see why FIPS is a concern for you.
Cheers, Michael.
On Mon, 8 Nov 2021 at 04:32, Michael Schroeder mls@suse.de wrote:
On Sat, Nov 06, 2021 at 07:43:02AM -0000, Daniel Alley wrote:
Another issue - which is not per-se a security issue but it's still a problem - is that deltarpm uses md5 checksums pervasively. They're everywhere. And it uses its own implementation of md5 which doesn't respect FIPS, so even when the user has *explicitly* configured their system to not use md5 for anything security-relevant, libdeltarpm won't know or care.
They are used as a consistency check, it might as well use crc32. So I don't see why FIPS is a concern for you.
In order to get the overall system to be FIPS (and equivalent EU/RU/CN ones) certified all the implementations of various functions have to be audited and reviewed. Some must be able to be turned off no matter what. It doesn't matter if 99 of the 100 versions of md5um are only for consistency, they must be able to be turned off/not used and not affect the system. [ The reason why we can't have nice things is that various super-programmers who see that 99 versions of md5sum are gone, but find that one call in say librpm which still exists, so they make a wrapper to it and then tie the bank code to it. Next thing you know, you find yourself not just on the Register as a story about code gone wrong but on the front page of various financial newspapers due to bank losses.]
Cheers, Michael.
-- Michael Schroeder SUSE Software Solutions Germany GmbH mls@suse.de GF: Felix Imendoerffer HRB 36809, AG Nuernberg main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);} _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Stephen John Smoogen smooge@gmail.com writes:
On Mon, 8 Nov 2021 at 04:32, Michael Schroeder mls@suse.de wrote:
On Sat, Nov 06, 2021 at 07:43:02AM -0000, Daniel Alley wrote:
Another issue - which is not per-se a security issue but it's still a problem - is that deltarpm uses md5 checksums pervasively. They're everywhere. And it uses its own implementation of md5 which doesn't respect FIPS, so even when the user has *explicitly* configured their system to not use md5 for anything security-relevant, libdeltarpm won't know or care.
They are used as a consistency check, it might as well use crc32. So I don't see why FIPS is a concern for you.
In order to get the overall system to be FIPS (and equivalent EU/RU/CN ones) certified all the implementations of various functions have to be audited and reviewed. Some must be able to be turned off no matter what. It doesn't matter if 99 of the 100 versions of md5um are only for consistency, they must be able to be turned off/not used and not affect the system.
I don't think that's quite accuroate. If the crypto primitive isn't being used for security, then FIPS isn't interested - FIPS is only certifying the cryptography used, and this isn't it. (It's non-FIPS relevant.)
This leads to a very common workaround for legacy cryptosystems of tunneling the "bad" crypto in something else: one example is interacting with RC4 and NTLM, where they're still used but over a tunnel (TLS, VPN, etc.) that doesn't expose them.
Be well, --Robbie
On Sat, 2021-11-06 at 07:43 +0000, Daniel Alley wrote:
On Wed, Aug 11, 2021 at 10:03:50PM +0200, Marek Marczykowski- Górecki wrote: I do think we should drop drpms or make them more useful, but I don't think there's any security angle here. (see below)
drpms work by downloading the delta, then using it + the version you have installed to recreate the signed rpm (just like you downloaded the full signed update) and then the gpg signature is checked of that full rpm, just like one you downloaded. If the drpm is tampered with it won't reassemble and it will fall back to the full signed rpm.
Sorry to resurrect this thread.
Another issue - which is not per-se a security issue but it's still a problem - is that deltarpm uses md5 checksums pervasively. They're everywhere. And it uses its own implementation of md5 which doesn't respect FIPS, so even when the user has *explicitly* configured their system to not use md5 for anything security-relevant, libdeltarpm won't know or care.
md5 used as a checksum to only detect network transmission issues is not a problem, and is not under the purview of the FIPS certification.
As mentioned above the actual packages are still finally reassembled and the signature checked, so that is what matters in terms of security (those algorithms and computations need to be FIPS approved and the implementation certified).
HTH, Simo.
On 11/8/21 12:36 PM, Simo Sorce wrote:
On Sat, 2021-11-06 at 07:43 +0000, Daniel Alley wrote:
On Wed, Aug 11, 2021 at 10:03:50PM +0200, Marek Marczykowski- Górecki wrote: I do think we should drop drpms or make them more useful, but I don't think there's any security angle here. (see below)
drpms work by downloading the delta, then using it + the version you have installed to recreate the signed rpm (just like you downloaded the full signed update) and then the gpg signature is checked of that full rpm, just like one you downloaded. If the drpm is tampered with it won't reassemble and it will fall back to the full signed rpm.
Sorry to resurrect this thread.
Another issue - which is not per-se a security issue but it's still a problem - is that deltarpm uses md5 checksums pervasively. They're everywhere. And it uses its own implementation of md5 which doesn't respect FIPS, so even when the user has *explicitly* configured their system to not use md5 for anything security-relevant, libdeltarpm won't know or care.
md5 used as a checksum to only detect network transmission issues is not a problem, and is not under the purview of the FIPS certification.
As mentioned above the actual packages are still finally reassembled and the signature checked, so that is what matters in terms of security (those algorithms and computations need to be FIPS approved and the implementation certified).
This is enough for FIPS, yes, but it is still very risky, as a bug in the package reassembly process is a remote root exploit. At a minimum, I recommend disabling deltarpm by default if repo_gpgcheck is not 1.
Sincerely, Demi Marie Obenour
Yeah rsync would be nice for instead at least for repository refreshes. No?
Am 11.08.21 um 22:03 schrieb Marek Marczykowski-Górecki:
- there is also argument that people's connection bandwidth nowadays tends to be fast enough to make the package rebuilding actually slower than downloading the whole package (but that really vary between different installations)
Since we now have Linux on smartphonesd, i heavily disagree with this assumption and advocate to expand the drpm usage.
best regards, Marius Schwarz
On 11/6/21 15:46, Marius Schwarz wrote:
Am 11.08.21 um 22:03 schrieb Marek Marczykowski-Górecki:
- there is also argument that people's connection bandwidth nowadays
tends to be fast enough to make the package rebuilding actually slower than downloading the whole package (but that really vary between different installations)
Since we now have Linux on smartphonesd, i heavily disagree with this assumption and advocate to expand the drpm usage.
Fedora on a smartphone is a very rare and niche case. It's something that can be considered, but should not be a driver for the distro's technology choices. Regardless, as the other points mentioned, I have not seen any significant download saving from it anyway.
On 11/6/21 7:42 PM, Samuel Sieb wrote:
On 11/6/21 15:46, Marius Schwarz wrote:
Am 11.08.21 um 22:03 schrieb Marek Marczykowski-Górecki:
- there is also argument that people's connection bandwidth nowadays
tends to be fast enough to make the package rebuilding actually slower than downloading the whole package (but that really vary between different installations)
Since we now have Linux on smartphonesd, i heavily disagree with this assumption and advocate to expand the drpm usage.
Fedora on a smartphone is a very rare and niche case. It's something that can be considered, but should not be a driver for the distro's technology choices. Regardless, as the other points mentioned, I have not seen any significant download saving from it anyway.
I have almost always seen it *increase* download times,
Sincerely,
Demi Marie Obenour (she/her/hers)
On Sun, Nov 7, 2021 at 2:22 AM Demi Marie Obenour demiobenour@gmail.com wrote:
I have almost always seen it *increase* download times,
In my experience, while the download times may be (slightly) reduced, on a number of my (slower) systems, the rebuild of the rpm itself took longer then it would have taken to download the full rpm.
As with all else, your mileage WILL vary.
On Sat, Nov 6, 2021 at 11:47 PM Marius Schwarz fedoradev@cloud-foo.de wrote:
Am 11.08.21 um 22:03 schrieb Marek Marczykowski-Górecki:
- there is also argument that people's connection bandwidth nowadays tends to be fast enough to make the package rebuilding actually slower than downloading the whole package (but that really vary between different installations)
Since we now have Linux on smartphonesd, i heavily disagree with this assumption and advocate to expand the drpm usage.
How would you suggest to do that?
From what I can tell, drpms are most useful if you update your system daily (because deltas are only kept from one compose, and need storage space) - but unless you have thousands of packages installed, the few hundred KB - a few MB of download savings you get from drpms every day (at most) are drowned out by the size of the daily repository metadata download to even get those drpms. And if you update less often to reduce the frequency with which you need to download the big repo metadata, you might not get drpms for any updates at all.
So I don't think that there's *any* scenario where using drpms results in a net reduction in MBs that need to be downloaded over time, because either - failed drpm reconstructions cause whole packages to be redownloaded anyway, - very few packages have drpms in every update push, - drpms are only kept for a very limited amount of time and limited number of packages, - savings from drpm usage is on the order of at most a few MB per update per day, and less if not updating every day, but - repository metadata downloads are reaching ~100MB / day, depending on how many repos are enabled.
So ... am I missing something, or do drpms actually make things worse in *every* respect right now? - more MBs downloaded per update / per day in every scenario I can think of, - more computationally intensive to reconstruct locally, - needs compute time and storage space in Fedora build system, - makes compose process more complicated.
Fabio
On 11/8/21 5:02 AM, Fabio Valentini wrote:
On Sat, Nov 6, 2021 at 11:47 PM Marius Schwarz fedoradev@cloud-foo.de wrote:
Am 11.08.21 um 22:03 schrieb Marek Marczykowski-Górecki:
- there is also argument that people's connection bandwidth nowadays tends to be fast enough to make the package rebuilding actually slower than downloading the whole package (but that really vary between different installations)
Since we now have Linux on smartphonesd, i heavily disagree with this assumption and advocate to expand the drpm usage.
How would you suggest to do that?
From what I can tell, drpms are most useful if you update your system daily (because deltas are only kept from one compose, and need storage space) - but unless you have thousands of packages installed, the few hundred KB - a few MB of download savings you get from drpms every day (at most) are drowned out by the size of the daily repository metadata download to even get those drpms. And if you update less often to reduce the frequency with which you need to download the big repo metadata, you might not get drpms for any updates at all.
So I don't think that there's *any* scenario where using drpms results in a net reduction in MBs that need to be downloaded over time, because either
- failed drpm reconstructions cause whole packages to be redownloaded anyway,
- very few packages have drpms in every update push,
- drpms are only kept for a very limited amount of time and limited
number of packages,
- savings from drpm usage is on the order of at most a few MB per
update per day, and less if not updating every day, but
- repository metadata downloads are reaching ~100MB / day, depending
on how many repos are enabled.
So ... am I missing something, or do drpms actually make things worse in *every* respect right now?
- more MBs downloaded per update / per day in every scenario I can think of,
- more computationally intensive to reconstruct locally,
- needs compute time and storage space in Fedora build system,
- makes compose process more complicated.
Fabio
I don’t think you are missing anything.
Sincerely, Demi Marie Obenour
On Sat, 2021-11-06 at 23:46 +0100, Marius Schwarz wrote:
Am 11.08.21 um 22:03 schrieb Marek Marczykowski-Górecki:
- there is also argument that people's connection bandwidth
nowadays tends to be fast enough to make the package rebuilding actually slower than downloading the whole package (but that really vary between different installations)
Since we now have Linux on smartphonesd, i heavily disagree with this assumption and advocate to expand the drpm usage.
I'm afraid that in its current implementation deltarpm is simply not very efficient, I usually see <5% saved when doing updates on my Fedora laptop. I would not expect this to differ too much from a a Fedora running on eq. PinePhone or other mobile device.
What could work well already is using ostree or ostree based Fedora distro on a phone - that has efficient delta-based updates built in and used by default. So using say Silverblue/Kinoite or similar on a phone could be much more efficient than using deltarpm + you get additional benefits of being able to roll back at any time, which could be useful on such a new Fedora platform.
Also flatpaks are ostree based & should get the same delta download and de-duplication benefits automatically. So using as many flatpaks as possible on a Fedora running mobile phone could also help quite a bit.
best regards, Marius Schwarz _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
-
Daniel Alley -
Demi M. Obenour -
Demi Marie Obenour
-
Fabio Valentini -
Gary Buhrmaster -
Kevin Fenzi -
Marek Marczykowski-Górecki -
Marius Schwarz -
Michael Schroeder -
mkolman＠redhat.com -
Neal Gompa -
Reon Beon -
Robbie Harwood -
Samuel Sieb -
Simo Sorce -
Stephen John Smoogen