Doing yum update cause yum to issue this message Finishing rebuild of rpms, from deltarpms
What is the significance of this, and why is it being done?
On 04/15/2011 03:42 PM, JD wrote:
Doing yum update cause yum to issue this message Finishing rebuild of rpms, from deltarpms
What is the significance of this, and why is it being done?
Presumably, you have the presto plugin installed and active. This means that instead of downloading the complete rpm for each update, yum just downloads the deltarpm, which only contains the changes from the last version, then applies the changes to the cached copies of the rpms, changing them to the latest version before updating. As most of the updated rpm is the same as the older version, this drops the amount of data downloaded by anywhere from 70-90%, sometimes more. Of course, people on unlimited data plans may not worry about the extra bandwidth, but very often the time taken to rebuild the rpms is less than the time needed to download the complete new version.
If the above doesn't quite make sense, don't worry; yum is just telling you that it's just finished saving you some (possibly valuable) bandwidth.
On 04/15/2011 04:44 PM, Joe Zeff wrote:
On 04/15/2011 03:42 PM, JD wrote:
Doing yum update cause yum to issue this message Finishing rebuild of rpms, from deltarpms
What is the significance of this, and why is it being done?
Presumably, you have the presto plugin installed and active. This means that instead of downloading the complete rpm for each update, yum just downloads the deltarpm, which only contains the changes from the last version, then applies the changes to the cached copies of the rpms, changing them to the latest version before updating. As most of the updated rpm is the same as the older version, this drops the amount of data downloaded by anywhere from 70-90%, sometimes more. Of course, people on unlimited data plans may not worry about the extra bandwidth, but very often the time taken to rebuild the rpms is less than the time needed to download the complete new version.
If the above doesn't quite make sense, don't worry; yum is just telling you that it's just finished saving you some (possibly valuable) bandwidth.
Sure it makes sense! So, only the items (files) of the rpm that changed get downloaded (my guess)?? That's not too bad. What would be better is getting a program to do a binary diff between old rpm and new rpm, producing a binary patch file, and applying the binary patch file on the user's old cache of the rpm and installing that. See http://www.daemonology.net/bsdiff/ and of course, bsdiff and bspatch are in the fedora repo as bsdiff-4.3-7.fc12.i686
Cheers,
JD
On 04/15/2011 09:04 PM, Andre Robatino wrote:
http://cedarandthistle.wordpress.com/2009/11/06/on-binary-delta-algorithms/
Read the article and parts of the thesis. So, the final version of the algorithm still remains with the University of Oxford??
I downloaded the bsdiff srpm for fc15 and built it.
I then used it to diff -rwxr-xr-x 1 root root 3699072 Feb 6 23:09 /boot/vmlinuz-2.6.35.11-83.fc14.i686 vs -rwxr-xr-x 1 root root 3702144 Mar 31 15:26 /boot/vmlinuz-2.6.35.12-88.fc14.i686 and the resulting patch -rw-r--r-- 1 root root 3554921 Apr 15 22:17 vmlinux-2.6.35.11-83--2.6.35.12-88.patch
So, you can see that the patch is almost the same size as the files diffed.
Obviously, version 4.3-8 does not contain the unpublished optimizations that Colin Percival mentions in his thesis.
On Fri, 2011-04-15 at 22:33 -0700, JD wrote:
On 04/15/2011 09:04 PM, Andre Robatino wrote:
http://cedarandthistle.wordpress.com/2009/11/06/on-binary-delta-algorithms/
Read the article and parts of the thesis. So, the final version of the algorithm still remains with the University of Oxford??
I downloaded the bsdiff srpm for fc15 and built it.
I then used it to diff -rwxr-xr-x 1 root root 3699072 Feb 6 23:09 /boot/vmlinuz-2.6.35.11-83.fc14.i686 vs -rwxr-xr-x 1 root root 3702144 Mar 31 15:26 /boot/vmlinuz-2.6.35.12-88.fc14.i686 and the resulting patch -rw-r--r-- 1 root root 3554921 Apr 15 22:17 vmlinux-2.6.35.11-83--2.6.35.12-88.patch
So, you can see that the patch is almost the same size as the files diffed.
Obviously, version 4.3-8 does not contain the unpublished optimizations that Colin Percival mentions in his thesis.
A couple of things:
1. vmlinuz is compressed and it's essentially useless to delta compressed files without uncompressing them first. 2. We use the program deltarpm to make deltas between rpms. Deltarpm uses a modified bsdiff algorithm as well as the add block enhancements mentioned in my blog post.
Jonathan
On 04/15/2011 10:58 PM, Jonathan Dieter wrote:
On Fri, 2011-04-15 at 22:33 -0700, JD wrote:
On 04/15/2011 09:04 PM, Andre Robatino wrote:
http://cedarandthistle.wordpress.com/2009/11/06/on-binary-delta-algorithms/
Read the article and parts of the thesis. So, the final version of the algorithm still remains with the University of Oxford??
I downloaded the bsdiff srpm for fc15 and built it.
I then used it to diff -rwxr-xr-x 1 root root 3699072 Feb 6 23:09 /boot/vmlinuz-2.6.35.11-83.fc14.i686 vs -rwxr-xr-x 1 root root 3702144 Mar 31 15:26 /boot/vmlinuz-2.6.35.12-88.fc14.i686 and the resulting patch -rw-r--r-- 1 root root 3554921 Apr 15 22:17 vmlinux-2.6.35.11-83--2.6.35.12-88.patch
So, you can see that the patch is almost the same size as the files diffed.
Obviously, version 4.3-8 does not contain the unpublished optimizations that Colin Percival mentions in his thesis.
A couple of things:
1. vmlinuz is compressed and it's essentially useless to delta compressed files without uncompressing them first. 2. We use the program deltarpm to make deltas between rpms. Deltarpm uses a modified bsdiff algorithm as well as the add block enhancements mentioned in my blog post.Jonathan
So, deltarpm actually uses the enhanced (i.e. modified) algorithm that Colin Percival mentions in his thesis, or is it from a different author?
I
On Fri, 2011-04-15 at 23:06 -0700, JD wrote:
On 04/15/2011 10:58 PM, Jonathan Dieter wrote:
On Fri, 2011-04-15 at 22:33 -0700, JD wrote:
On 04/15/2011 09:04 PM, Andre Robatino wrote:
http://cedarandthistle.wordpress.com/2009/11/06/on-binary-delta-algorithms/
Read the article and parts of the thesis. So, the final version of the algorithm still remains with the University of Oxford??
I downloaded the bsdiff srpm for fc15 and built it.
I then used it to diff -rwxr-xr-x 1 root root 3699072 Feb 6 23:09 /boot/vmlinuz-2.6.35.11-83.fc14.i686 vs -rwxr-xr-x 1 root root 3702144 Mar 31 15:26 /boot/vmlinuz-2.6.35.12-88.fc14.i686 and the resulting patch -rw-r--r-- 1 root root 3554921 Apr 15 22:17 vmlinux-2.6.35.11-83--2.6.35.12-88.patch
So, you can see that the patch is almost the same size as the files diffed.
Obviously, version 4.3-8 does not contain the unpublished optimizations that Colin Percival mentions in his thesis.
A couple of things:
1. vmlinuz is compressed and it's essentially useless to delta compressed files without uncompressing them first. 2. We use the program deltarpm to make deltas between rpms. Deltarpm uses a modified bsdiff algorithm as well as the add block enhancements mentioned in my blog post.Jonathan
So, deltarpm actually uses the enhanced (i.e. modified) algorithm that Colin Percival mentions in his thesis, or is it from a different author?
Yes, though a different person wrote the code.
Jonathan
On 04/15/2011 06:49 PM, JD wrote:
What would be better is getting a program to do a binary diff between old rpm and new rpm, producing a binary patch file, and applying the binary patch file on the user's old cache of the rpm and installing that.
That's exactly what a deltarpm is. How many files do you think there are to be changed when you download a new kernel, for example.
-
Andre Robatino -
JD -
Joe Zeff -
Jonathan Dieter