Is it possible to make a deltaiso without having both the older ISO and the Newer ISO on a local system.
Example. Fedora-14-Alpha-x86_64-Live.iso was downloaded. A Fedora-14-Alpha-2-x86_64-Live.iso is availible for download.
Can a deltaiso be made without downloading the newer ISO?
makedeltaiso Fedora-14-Alpha-x86_64-Live.iso http://path/to/folder/containing/Fedora-14-Alpha-2-x86_64-Live.iso deltaiso
On Sun, 2010-08-29 at 01:49 -0400, David wrote:
Is it possible to make a deltaiso without having both the older ISO and the Newer ISO on a local system.
Example. Fedora-14-Alpha-x86_64-Live.iso was downloaded. A Fedora-14-Alpha-2-x86_64-Live.iso is availible for download.
Can a deltaiso be made without downloading the newer ISO?
makedeltaiso Fedora-14-Alpha-x86_64-Live.iso http://path/to/folder/containing/Fedora-14-Alpha-2-x86_64-Live.iso deltaiso
No. Makedeltaiso doesn't understand web links, only local files.
Jonathan
On 8/29/2010 2:30 AM, Jonathan Dieter wrote:
On Sun, 2010-08-29 at 01:49 -0400, David wrote:
Is it possible to make a deltaiso without having both the older ISO and the Newer ISO on a local system.
Example. Fedora-14-Alpha-x86_64-Live.iso was downloaded. A Fedora-14-Alpha-2-x86_64-Live.iso is availible for download.
Can a deltaiso be made without downloading the newer ISO?
makedeltaiso Fedora-14-Alpha-x86_64-Live.iso http://path/to/folder/containing/Fedora-14-Alpha-2-x86_64-Live.iso deltaiso
No. Makedeltaiso doesn't understand web links, only local files.
Too bad. The delta concept, both rpm and ISO, is very useful and functions very well IMO.
Perhaps added in the future as an enhancement? :-)
On Sun, 2010-08-29 at 06:51 -0400, David wrote:
On 8/29/2010 2:30 AM, Jonathan Dieter wrote:
On Sun, 2010-08-29 at 01:49 -0400, David wrote:
Is it possible to make a deltaiso without having both the older ISO and the Newer ISO on a local system.
Example. Fedora-14-Alpha-x86_64-Live.iso was downloaded. A Fedora-14-Alpha-2-x86_64-Live.iso is availible for download.
Can a deltaiso be made without downloading the newer ISO?
makedeltaiso Fedora-14-Alpha-x86_64-Live.iso http://path/to/folder/containing/Fedora-14-Alpha-2-x86_64-Live.iso deltaiso
No. Makedeltaiso doesn't understand web links, only local files.
Too bad. The delta concept, both rpm and ISO, is very useful and functions very well IMO.
Perhaps added in the future as an enhancement? :-)
Patches are always welcome. :)
Jonathan
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 8/29/2010 8:05 AM, Jonathan Dieter wrote:
On Sun, 2010-08-29 at 06:51 -0400, David wrote:
On 8/29/2010 2:30 AM, Jonathan Dieter wrote:
On Sun, 2010-08-29 at 01:49 -0400, David wrote:
Is it possible to make a deltaiso without having both the older ISO and the Newer ISO on a local system.
Example. Fedora-14-Alpha-x86_64-Live.iso was downloaded. A Fedora-14-Alpha-2-x86_64-Live.iso is availible for download.
Can a deltaiso be made without downloading the newer ISO?
makedeltaiso Fedora-14-Alpha-x86_64-Live.iso http://path/to/folder/containing/Fedora-14-Alpha-2-x86_64-Live.iso deltaiso
No. Makedeltaiso doesn't understand web links, only local files.
Too bad. The delta concept, both rpm and ISO, is very useful and functions very well IMO.
Perhaps added in the future as an enhancement? :-)
Patches are always welcome. :)
:-) I'm sure that they are.
However... my programing skills ended long ago with GWBasic. And even then nothing anywhere near as impressive as what you developers write today. - --
David
David <dgboles <at> gmail.com> writes:
Is it possible to make a deltaiso without having both the older ISO and the Newer ISO on a local system.
Example. Fedora-14-Alpha-x86_64-Live.iso was downloaded. A Fedora-14-Alpha-2-x86_64-Live.iso is availible for download.
Can a deltaiso be made without downloading the newer ISO?
makedeltaiso Fedora-14-Alpha-x86_64-Live.iso http://path/to/folder/containing/Fedora-14-Alpha-2-x86_64-Live.iso deltaiso
Deltaisos won't save significant space when used on Live images. Having said that, if you were able to do the above (similar to what the rpm command can do), you would still be downloading the ISO, just not storing it on disk. Disk space is cheap, and as long as you have a few spare gig of space, you could just download the ISO, generate the diso, then keep the diso and delete the ISO to save on disk space. If you're using disos to save on disk space, you probably also want to verify that they reconstruct properly (which will require some temporary disk space, anyway) and to have multiple backups (since if you lose one, you can't reconstruct any of the ISOs after that).
On 8/29/2010 12:39 PM, Andre Robatino wrote:
David <dgboles <at> gmail.com> writes:
Is it possible to make a deltaiso without having both the older ISO and the Newer ISO on a local system.
Example. Fedora-14-Alpha-x86_64-Live.iso was downloaded. A Fedora-14-Alpha-2-x86_64-Live.iso is availible for download.
Can a deltaiso be made without downloading the newer ISO?
makedeltaiso Fedora-14-Alpha-x86_64-Live.iso http://path/to/folder/containing/Fedora-14-Alpha-2-x86_64-Live.iso deltaiso
Deltaisos won't save significant space when used on Live images. Having said that, if you were able to do the above (similar to what the rpm command can do), you would still be downloading the ISO, just not storing it on disk. Disk space is cheap, and as long as you have a few spare gig of space, you could just download the ISO, generate the diso, then keep the diso and delete the ISO to save on disk space. If you're using disos to save on disk space, you probably also want to verify that they reconstruct properly (which will require some temporary disk space, anyway) and to have multiple backups (since if you lose one, you can't reconstruct any of the ISOs after that).
I think that I understand the principal behind this. Disk space nor bandwidth are not problems for me. I was thinking of others that might have a bandwidth/byte counter problem.
Explain to me just how downloading only a part of the new ISO, the changes, and then creating the new one with parts of the old ISO and the new parts. How can that be the same as downloading the whole new ISO?
Or am I not understanding how this works.
On Sun, 2010-08-29 at 14:29 -0400, David wrote:
I think that I understand the principal behind this. Disk space nor bandwidth are not problems for me. I was thinking of others that might have a bandwidth/byte counter problem.
Explain to me just how downloading only a part of the new ISO, the changes, and then creating the new one with parts of the old ISO and the new parts. How can that be the same as downloading the whole new ISO?
Or am I not understanding how this works.
The steps in a deltaiso lifecycle are as follows:
The Fedora project (or an interested user) creates the deltaiso. To do so, they need the old iso and the new iso, and they generate the deltaiso using makedeltaiso. So, if we wanted to create a deltaiso from F14-Alpha to F14-Beta, we might do something like:
makedeltaiso F14-Alpha.iso F14-Beta.iso F14-Alpha_Beta.diso
This deltaiso will contain the *difference* from F14-Alpha to F14-Beta.
A user who has the iso for F14-Alpha (and low bandwidth) will then download F14-Alpha_Beta.diso and run something like:
applydeltaiso F14-Alpha.iso F14-Alpha_Beta.diso F14-Beta.iso
They will then "magically" get F14-Beta.iso without having to download it. Of course, applying the deltaiso will take a long time, so it really only makes sense to use it if you're on a limited bandwidth connection.
Please note that normal users should never need to *make* the deltaiso, they should just *apply* them. And Andre's done a great job of making deltaiso's available, especially during prerelease.
Jonathan
On Sun, 2010-08-29 at 21:56 +0300, Jonathan Dieter wrote:
The Fedora project (or an interested user) creates the deltaiso.
To clarify, this is theoretical. The Fedora project is *not* creating official deltaisos right now. However, any deltaiso should build into a byte-for-byte copy of the new iso, which can then be verified using normal methods.
Jonathan
On 8/29/2010 2:56 PM, Jonathan Dieter wrote:
On Sun, 2010-08-29 at 14:29 -0400, David wrote:
I think that I understand the principal behind this. Disk space nor bandwidth are not problems for me. I was thinking of others that might have a bandwidth/byte counter problem.
Explain to me just how downloading only a part of the new ISO, the changes, and then creating the new one with parts of the old ISO and the new parts. How can that be the same as downloading the whole new ISO?
Or am I not understanding how this works.
The steps in a deltaiso lifecycle are as follows:
The Fedora project (or an interested user) creates the deltaiso. To do so, they need the old iso and the new iso, and they generate the deltaiso using makedeltaiso. So, if we wanted to create a deltaiso from F14-Alpha to F14-Beta, we might do something like:
makedeltaiso F14-Alpha.iso F14-Beta.iso F14-Alpha_Beta.diso
This deltaiso will contain the *difference* from F14-Alpha to F14-Beta.
A user who has the iso for F14-Alpha (and low bandwidth) will then download F14-Alpha_Beta.diso and run something like:
applydeltaiso F14-Alpha.iso F14-Alpha_Beta.diso F14-Beta.iso
They will then "magically" get F14-Beta.iso without having to download it. Of course, applying the deltaiso will take a long time, so it really only makes sense to use it if you're on a limited bandwidth connection.
Please note that normal users should never need to *make* the deltaiso, they should just *apply* them. And Andre's done a great job of making deltaiso's available, especially during prerelease.
Jonathan
I have used this during Fedora 13 but only to see how it works. Not working was never a thought. :-)
What you just explained is how I had understood this working.
And yes Andre has done an excleant job of making and providing deltaisos.
What I was refering to was that someone said that having the origianl ISO and then downloading the deltaiso and making the new ISO would be the same as downloading the whole new ISO. That is what i was questioning.
David <dgboles <at> gmail.com> writes:
I think that I understand the principal behind this. Disk space nor bandwidth are not problems for me. I was thinking of others that might have a bandwidth/byte counter problem.
Explain to me just how downloading only a part of the new ISO, the changes, and then creating the new one with parts of the old ISO and the new parts. How can that be the same as downloading the whole new ISO?
Or am I not understanding how this works.
If what you're asking is, is there a way of getting the bandwidth savings of deltaisos without someone else having to provide them, then no, I don't think so. Deltaisos basically work the same way as deltarpms, just on entire ISOs instead of individual RPMs. If there was a way to do that, then it wouldn't be necessary for deltarpms to be generated in order for yum-presto to work, either. The delta compression needed to express the changes between the old and new files is expensive, and if it wasn't done once and stored in a file (a deltaiso or a deltarpm), then the server would have to do it separately for each download, which would be prohibitive.
The processing that rsync/zsync does is much simpler, but generally doesn't provide as much bandwidth savings as deltaisos. Unfortunately, even rsync is expensive on the server, so not all servers provide it. And zsync, which puts the load on the client instead of the server, isn't available in Fedora yet (though you can install it from other sources), and like deltaisos, someone else has to provide zsync control files before it can be used. I've been doing this for recent test releases at
https://fedoraproject.org/wiki/User:Robatino/Downloads
so people can experiment with it and maybe create some more pressure to get it into Fedora. See
https://bugzilla.redhat.com/show_bug.cgi?id=490140 https://bugzilla.redhat.com/show_bug.cgi?id=495310
The performance of zsync vs. deltaisos varies from almost as good to much worse, depending on how many packages have changed between old and new ISOs. (For example, going from Fedora (N-1) to Fedora N Alpha TC1, zsync is almost useless, since almost all packages are updated. Between later iterations, it helps a lot.) Even if zsync becomes available, deltaisos should still be provided for people with very slow connections.
Should also mention that since the difference between zsync and deltaisos is either downloading changed RPMs in full vs. using deltarpms on them, this means that if both were provided, then people with either sufficiently fast connections (meaning around 5 Mbits/s) or a very slow PC would be better off using zsync (since downloading a full RPM is faster for them than downloading the deltarpm and applying it), while others would be better off with deltaisos. In fact, it should be exactly the same people who find that updates are faster with yum-presto disabled (download full RPMs) vs. enabled (download deltarpms and then apply them).
Since yum-presto is currently enabled by default, I think deltaisos should be provided just for consistency. Having high bandwidth has nothing to do with being a good tester, so there's no reason why testers should be expected to have more bandwidth than ordinary users.
2010/8/29 Andre Robatino robatino@fedoraproject.org: <--SNIP-->
The performance of zsync vs. deltaisos varies from almost as good to much worse, depending on how many packages have changed between old and new ISOs. (For example, going from Fedora (N-1) to Fedora N Alpha TC1, zsync is almost useless, since almost all packages are updated. Between later iterations, it helps a lot.) Even if zsync becomes available, deltaisos should still be provided for people with very slow connections.
One more question about delta' theory. Does Fedora-project host deltarpms for every possible package versions or only for the version that precedes the current version of a package? I mean that if one tries to update N-2 version of some package to its N' version has he to download an rpm or there's a drpm for all currently supported Fedora versions?
On 8/29/2010 3:14 PM, Andre Robatino wrote:
David <dgboles <at> gmail.com> writes:
I think that I understand the principal behind this. Disk space nor bandwidth are not problems for me. I was thinking of others that might have a bandwidth/byte counter problem.
Explain to me just how downloading only a part of the new ISO, the changes, and then creating the new one with parts of the old ISO and the new parts. How can that be the same as downloading the whole new ISO?
Or am I not understanding how this works.
If what you're asking is, is there a way of getting the bandwidth savings of deltaisos without someone else having to provide them, then no, I don't think so. Deltaisos basically work the same way as deltarpms, just on entire ISOs instead of individual RPMs. If there was a way to do that, then it wouldn't be necessary for deltarpms to be generated in order for yum-presto to work, either. The delta compression needed to express the changes between the old and new files is expensive, and if it wasn't done once and stored in a file (a deltaiso or a deltarpm), then the server would have to do it separately for each download, which would be prohibitive.
No sir. What my question was... Someone said that downloading the deltaiso and making the 'new' ISO from the delta and the original ISO was the same as just downloading the new ISO. I felt that was incorrect or I just did not understand this.
The processing that rsync/zsync does is much simpler, but generally doesn't provide as much bandwidth savings as deltaisos. Unfortunately, even rsync is expensive on the server, so not all servers provide it. And zsync, which puts the load on the client instead of the server, isn't available in Fedora yet (though you can install it from other sources), and like deltaisos, someone else has to provide zsync control files before it can be used. I've been doing this for recent test releases at
https://fedoraproject.org/wiki/User:Robatino/Downloads
so people can experiment with it and maybe create some more pressure to get it into Fedora. See
https://bugzilla.redhat.com/show_bug.cgi?id=490140 https://bugzilla.redhat.com/show_bug.cgi?id=495310
The performance of zsync vs. deltaisos varies from almost as good to much worse, depending on how many packages have changed between old and new ISOs. (For example, going from Fedora (N-1) to Fedora N Alpha TC1, zsync is almost useless, since almost all packages are updated. Between later iterations, it helps a lot.) Even if zsync becomes available, deltaisos should still be provided for people with very slow connections.
And BTW thank you very much for the deltaiso packages. I am sure that they are appreciated.
-
Andre Robatino -
David -
Hiisi -
Jonathan Dieter