[fedora-arm] builder io isue

[Gordan Bobic]

On 12/26/2011 07:34 PM, Dennis Gilmore wrote:

>>>>> Why not just mount direct via NFS? It'd be a lot quicker, not to
>>>>> mention easier to tune. It'd work for building all but a handful
>>>>> of packages (e.g. zsh), but you could handle that by having a
>>>>> single builder that uses a normal fs that has a policy pointing
>>>>> the packages that fail self-tests on NFS at it.
>>>>
>>>> I'm not acquainted with the rationale for the decision so perhaps
>>>> somebody else can comment.  Beyond the packages that demand a local
>>>> filesystem, perhaps there were issues with .nfsXXX files, or some
>>>> stability problem not seen when working with a single open file?
>>>> Not sure.
>>>
>>> glibc uses functionality not supported on nfsv3.
>>
>> That may be so, but it does not appear to be in any way an issue for
>> having mock chroot on NFS. It also causes no problems with running on
>> NFS root. Can you provide a single relevant example of why this is
>> relevant for /var/lib/mock?
>
> * Mon Feb 14 2011 Andreas Schwab<schwab at redhat.com>  - 2.13.90-4
> - Update from master
>    - Update sysdeps/unix/sysv/linux/sparc/bits/socket.h
>    - Synchronize generic bits/sched.h cpu_set_t with Linux implementation
>    - Schedule nscd cache pruning more accurately from re-added values
>    - Fix passing symbol value to pltexit callbacks when ld.so auditing
>    - Fix range error handling in sgetspent
> - Revert "Fix ordering of DSO constructors and destructors" (#673014)
> - Create debuginfo-common on biarch archs
> - Reinstall assembler workaround.
> - Replace setuid by file capabilities (#646469)
>
> thats the change that makes it not work,  nfsv3 doesnt support file
> capablities. initing a chroot fails. which means you can not use mock
> on it.

Interesting. Judging by the version and date, this isn't the case in 
either RHEL6 nor F13 - which would explain why I haven't seen it.

Thanks for pointing this out. :)

>>> nfsv4 may be a viable
>>> option. mock doesnt work right on nfs.
>>
>> Can you provide an example? I certainly found no problem with it. The
>> only issue I had turned out to be caused by the file system that was
>> being exported via NFS having a bug that made the sticky directories
>> not work properly. Ever since I fixed that it's been working
>> beautifully.
 >
> rhel6 is too old to hit the issue.

If it breaks functionality that used to work, isn't that technically a 
regression bug? ;)

>>> i think looking at iscsi or aoe
>>> or nbd would help to take out some of the layering thats in play
>>> currently. I really think we need to get more spindles in play. all
>>> the other arches use raid0 over local scsi/sas drives that are 10k
>>> or 15k rpm. and the storage is local.  the only local option we
>>> have is to use USB storage. maybe a 16GB usb stick on each may
>>> help. they can be gotten for ~USD$16 each we are still going to be
>>> limited to the usb bus.
>>
>> Been there, tried it. Most USB sticks have similar random-write IOPS
>> performancs to SD cards (i.e. somewhere between dire and useless).
>> The ones that I have found that fare bearably well (i.e. come close
>> to a ~ 5000 rpm disk) are _some_ of the ones based on a Kingston
>> controller, but with the generic no-name sticks you will have to test
>> a large selection of models to find a model that is any good.
>
> I never said it was perfect. just that it may be better that the io
> bound bottle neck we have right now.

Hmm...
Assuming 20 IOPS for an average SD card or USB stick and 120 IOPS from 1 
7200rpm disk:

40x builders * 20 IOPS = 800 IOPS
4x disks * 120 IOPS = 480 IOPS

Indeed, it sounds like you might plausibly end up ahead. But 40x good 
16GB Class 10 SD cards isn't likely to be the most cost effective option.

Gordan