fedup performance

[Panu Matilainen]

On 07/03/2013 11:29 AM, Alex G. wrote:
> On 07/03/2013 03:23 AM, Panu Matilainen wrote:
>> On 07/03/2013 09:59 AM, Panu Matilainen wrote:
>>> On 07/03/2013 07:42 AM, Alex G. wrote:
>>>> On 07/02/2013 08:28 PM, Neal Becker wrote:
>>>>> Not d/l speed related.  I just want to share.  I update a very fast 8
>>>>> core
>>>>> server, with a conventional disk drive.  Took 2-3 hours, not
>>>>> including d/l.
>>>>>
>>>>> I update my laptop which has an ssd (and MORE packages).  Took 10-15
>>>>> minutes.
>>>>>
>>>> I think this might simply have to do with rpm running ldconfig (a very
>>>> disk IO expensive operation) for a large number of packages. I'm not
>>>> sure yum/rpm has deferred ldconfig processing.
>>>>
>>>> DISCLAIMER: I may be very wrong. Please don't quote me on this.
>>>
>>> ldconfig gets run a lot yes, but its also really fast these days.
>>> fdatasync() which gets called even more (a lot more at that) seems like
>>> a more likely painpoint on upgrades.
>>
>> Oh and here are some numbers for your entertainment. This is a 185 core
>> package install to empty chroot on my laptop with a conventional disk,
>> with the two worst script-offenders (kernel and selinux-policy-targeted
>> have) taken out of the picture as they'd very much dominate the running
>> time on a set this small:
>>
>> fdatasync, no scripts           1m16s
>> fdatasync, scripts              1m29s
>>
>> no fdatasync, no scripts          16s
>> no fdatasync, scripts             25s
>>
>> When fdatasync() is disabled (on initial install where there's no data
>> to lose), sure all the scripts start taking a considerable portion of
>> the running time. But for normal operation (such as upgrades),
>> fdatasync() is where the vast majority of time gets spent.
>>
>> Of course on real-world upgrades there are many many more things at play
>> than in the simple test-case above, but to improve performance you need
>> to figure out where the time is getting spent, guessing gets you nowhere.
>>
> Guessing gets other people to research the matter. A great way to get
> others to work for you for free. :p

Sometimes maybe, but it can also be a great way to irritate people who 
have done their research a long time ago.

	- Panu -