fedup performance

[Panu Matilainen]

On 07/03/2013 09:59 AM, Panu Matilainen wrote:
> On 07/03/2013 07:42 AM, Alex G. wrote:
>> On 07/02/2013 08:28 PM, Neal Becker wrote:
>>> Not d/l speed related.  I just want to share.  I update a very fast 8
>>> core
>>> server, with a conventional disk drive.  Took 2-3 hours, not
>>> including d/l.
>>>
>>> I update my laptop which has an ssd (and MORE packages).  Took 10-15
>>> minutes.
>>>
>> I think this might simply have to do with rpm running ldconfig (a very
>> disk IO expensive operation) for a large number of packages. I'm not
>> sure yum/rpm has deferred ldconfig processing.
>>
>> DISCLAIMER: I may be very wrong. Please don't quote me on this.
>
> ldconfig gets run a lot yes, but its also really fast these days.
> fdatasync() which gets called even more (a lot more at that) seems like
> a more likely painpoint on upgrades.

Oh and here are some numbers for your entertainment. This is a 185 core 
package install to empty chroot on my laptop with a conventional disk, 
with the two worst script-offenders (kernel and selinux-policy-targeted 
have) taken out of the picture as they'd very much dominate the running 
time on a set this small:

fdatasync, no scripts           1m16s
fdatasync, scripts              1m29s

no fdatasync, no scripts          16s
no fdatasync, scripts             25s

When fdatasync() is disabled (on initial install where there's no data 
to lose), sure all the scripts start taking a considerable portion of 
the running time. But for normal operation (such as upgrades), 
fdatasync() is where the vast majority of time gets spent.

Of course on real-world upgrades there are many many more things at play 
than in the simple test-case above, but to improve performance you need 
to figure out where the time is getting spent, guessing gets you nowhere.

	- Panu -

	- Panu -