Not d/l speed related. I just want to share. I update a very fast 8 core server, with a conventional disk drive. Took 2-3 hours, not including d/l.
I update my laptop which has an ssd (and MORE packages). Took 10-15 minutes.
On 07/02/2013 08:28 PM, Neal Becker wrote:
Not d/l speed related. I just want to share. I update a very fast 8 core server, with a conventional disk drive. Took 2-3 hours, not including d/l.
I update my laptop which has an ssd (and MORE packages). Took 10-15 minutes.
I think this might simply have to do with rpm running ldconfig (a very disk IO expensive operation) for a large number of packages. I'm not sure yum/rpm has deferred ldconfig processing.
DISCLAIMER: I may be very wrong. Please don't quote me on this.
Alex
On 2013-07-02 21:42, Alex G. wrote:
On 07/02/2013 08:28 PM, Neal Becker wrote:
Not d/l speed related. I just want to share. I update a very fast 8 core server, with a conventional disk drive. Took 2-3 hours, not including d/l.
I update my laptop which has an ssd (and MORE packages). Took 10-15 minutes.
I think this might simply have to do with rpm running ldconfig (a very disk IO expensive operation) for a large number of packages. I'm not sure yum/rpm has deferred ldconfig processing.
rpm has the concept of %posttrans . Stuff in %posttrans is run after *the entire transaction* has completed, not after *the specific package install* has completed.
However, I think we can't put ldconfig in %posttrans, because what happens if a package installed later relies on the ldconfig being correct for a package installed earlier in the same transaction?
DISCLAIMER: I may be very wrong. Please don't quote me on this.
Seconded =)
On 07/03/2013 12:15 AM, Adam Williamson wrote:
On 2013-07-02 21:42, Alex G. wrote:
On 07/02/2013 08:28 PM, Neal Becker wrote:
Not d/l speed related. I just want to share. I update a very fast 8 core server, with a conventional disk drive. Took 2-3 hours, not including d/l.
I update my laptop which has an ssd (and MORE packages). Took 10-15 minutes.
I think this might simply have to do with rpm running ldconfig (a very disk IO expensive operation) for a large number of packages. I'm not sure yum/rpm has deferred ldconfig processing.
rpm has the concept of %posttrans . Stuff in %posttrans is run after *the entire transaction* has completed, not after *the specific package install* has completed.
However, I think we can't put ldconfig in %posttrans, because what happens if a package installed later relies on the ldconfig being correct for a package installed earlier in the same transaction?
aptitude has something called "deferred ldconfig processing", and annoyingly, aptitude updates faster than yum. I've always wondered how yum/rpm can be smartized to speed things up this way. But this discussion is for a brighter day.
DISCLAIMER: I may be very wrong. Please don't quote me on this.
Seconded =)
My name is Alexandru Gagniuc, and I approve this message. :)
Alex
On 2013-07-02 22:25, Alex G. wrote:
On 07/03/2013 12:15 AM, Adam Williamson wrote:
On 2013-07-02 21:42, Alex G. wrote:
On 07/02/2013 08:28 PM, Neal Becker wrote:
Not d/l speed related. I just want to share. I update a very fast 8 core server, with a conventional disk drive. Took 2-3 hours, not including d/l.
I update my laptop which has an ssd (and MORE packages). Took 10-15 minutes.
I think this might simply have to do with rpm running ldconfig (a very disk IO expensive operation) for a large number of packages. I'm not sure yum/rpm has deferred ldconfig processing.
rpm has the concept of %posttrans . Stuff in %posttrans is run after *the entire transaction* has completed, not after *the specific package install* has completed.
However, I think we can't put ldconfig in %posttrans, because what happens if a package installed later relies on the ldconfig being correct for a package installed earlier in the same transaction?
aptitude has something called "deferred ldconfig processing", and annoyingly, aptitude updates faster than yum.
*mumble mumble* correlation *mumble* causation *mumble*
On Wed, Jul 03, 2013 at 12:25:19AM -0500, Alex G. wrote:
aptitude has something called "deferred ldconfig processing", and annoyingly, aptitude updates faster than yum. I've always wondered how yum/rpm can be smartized to speed things up this way. But this discussion is for a brighter day.
Well, this is how dpkg works. It downloads all packages, then "unpacks" it on filesystem and then "configures" them all, which means "running post-scripts". Personally, I dislike this design...
On 07/03/2013 07:42 AM, Alex G. wrote:
On 07/02/2013 08:28 PM, Neal Becker wrote:
Not d/l speed related. I just want to share. I update a very fast 8 core server, with a conventional disk drive. Took 2-3 hours, not including d/l.
I update my laptop which has an ssd (and MORE packages). Took 10-15 minutes.
I think this might simply have to do with rpm running ldconfig (a very disk IO expensive operation) for a large number of packages. I'm not sure yum/rpm has deferred ldconfig processing.
DISCLAIMER: I may be very wrong. Please don't quote me on this.
ldconfig gets run a lot yes, but its also really fast these days. fdatasync() which gets called even more (a lot more at that) seems like a more likely painpoint on upgrades.
- Panu -
On 07/03/2013 09:59 AM, Panu Matilainen wrote:
On 07/03/2013 07:42 AM, Alex G. wrote:
On 07/02/2013 08:28 PM, Neal Becker wrote:
Not d/l speed related. I just want to share. I update a very fast 8 core server, with a conventional disk drive. Took 2-3 hours, not including d/l.
I update my laptop which has an ssd (and MORE packages). Took 10-15 minutes.
I think this might simply have to do with rpm running ldconfig (a very disk IO expensive operation) for a large number of packages. I'm not sure yum/rpm has deferred ldconfig processing.
DISCLAIMER: I may be very wrong. Please don't quote me on this.
ldconfig gets run a lot yes, but its also really fast these days. fdatasync() which gets called even more (a lot more at that) seems like a more likely painpoint on upgrades.
Oh and here are some numbers for your entertainment. This is a 185 core package install to empty chroot on my laptop with a conventional disk, with the two worst script-offenders (kernel and selinux-policy-targeted have) taken out of the picture as they'd very much dominate the running time on a set this small:
fdatasync, no scripts 1m16s fdatasync, scripts 1m29s
no fdatasync, no scripts 16s no fdatasync, scripts 25s
When fdatasync() is disabled (on initial install where there's no data to lose), sure all the scripts start taking a considerable portion of the running time. But for normal operation (such as upgrades), fdatasync() is where the vast majority of time gets spent.
Of course on real-world upgrades there are many many more things at play than in the simple test-case above, but to improve performance you need to figure out where the time is getting spent, guessing gets you nowhere.
- Panu -
- Panu -
On 07/03/2013 03:23 AM, Panu Matilainen wrote:
On 07/03/2013 09:59 AM, Panu Matilainen wrote:
On 07/03/2013 07:42 AM, Alex G. wrote:
On 07/02/2013 08:28 PM, Neal Becker wrote:
Not d/l speed related. I just want to share. I update a very fast 8 core server, with a conventional disk drive. Took 2-3 hours, not including d/l.
I update my laptop which has an ssd (and MORE packages). Took 10-15 minutes.
I think this might simply have to do with rpm running ldconfig (a very disk IO expensive operation) for a large number of packages. I'm not sure yum/rpm has deferred ldconfig processing.
DISCLAIMER: I may be very wrong. Please don't quote me on this.
ldconfig gets run a lot yes, but its also really fast these days. fdatasync() which gets called even more (a lot more at that) seems like a more likely painpoint on upgrades.
Oh and here are some numbers for your entertainment. This is a 185 core package install to empty chroot on my laptop with a conventional disk, with the two worst script-offenders (kernel and selinux-policy-targeted have) taken out of the picture as they'd very much dominate the running time on a set this small:
fdatasync, no scripts 1m16s fdatasync, scripts 1m29s
no fdatasync, no scripts 16s no fdatasync, scripts 25s
When fdatasync() is disabled (on initial install where there's no data to lose), sure all the scripts start taking a considerable portion of the running time. But for normal operation (such as upgrades), fdatasync() is where the vast majority of time gets spent.
Of course on real-world upgrades there are many many more things at play than in the simple test-case above, but to improve performance you need to figure out where the time is getting spent, guessing gets you nowhere.
Guessing gets other people to research the matter. A great way to get others to work for you for free. :p
Alex
- Panu - - Panu -
On 07/03/2013 11:29 AM, Alex G. wrote:
On 07/03/2013 03:23 AM, Panu Matilainen wrote:
On 07/03/2013 09:59 AM, Panu Matilainen wrote:
On 07/03/2013 07:42 AM, Alex G. wrote:
On 07/02/2013 08:28 PM, Neal Becker wrote:
Not d/l speed related. I just want to share. I update a very fast 8 core server, with a conventional disk drive. Took 2-3 hours, not including d/l.
I update my laptop which has an ssd (and MORE packages). Took 10-15 minutes.
I think this might simply have to do with rpm running ldconfig (a very disk IO expensive operation) for a large number of packages. I'm not sure yum/rpm has deferred ldconfig processing.
DISCLAIMER: I may be very wrong. Please don't quote me on this.
ldconfig gets run a lot yes, but its also really fast these days. fdatasync() which gets called even more (a lot more at that) seems like a more likely painpoint on upgrades.
Oh and here are some numbers for your entertainment. This is a 185 core package install to empty chroot on my laptop with a conventional disk, with the two worst script-offenders (kernel and selinux-policy-targeted have) taken out of the picture as they'd very much dominate the running time on a set this small:
fdatasync, no scripts 1m16s fdatasync, scripts 1m29s
no fdatasync, no scripts 16s no fdatasync, scripts 25s
When fdatasync() is disabled (on initial install where there's no data to lose), sure all the scripts start taking a considerable portion of the running time. But for normal operation (such as upgrades), fdatasync() is where the vast majority of time gets spent.
Of course on real-world upgrades there are many many more things at play than in the simple test-case above, but to improve performance you need to figure out where the time is getting spent, guessing gets you nowhere.
Guessing gets other people to research the matter. A great way to get others to work for you for free. :p
Sometimes maybe, but it can also be a great way to irritate people who have done their research a long time ago.
- Panu -
-
Adam Williamson -
Alex Gagniuc -
Alexey I. Froloff -
Neal Becker -
Panu Matilainen