Performance

[Glen Eustace]

I have been having fun customising my kiosk CD.  I may have gone about
things the wrong way but I am making progress.

For most stuff I added a script to post_install_scripts
/usr/share/kadischi/post_install_scripts/99MyStuff.sh

I got a little caught out by some of the other scripts, 05fsclean.py for
example deletes everything from /root, when I had already put files into
using an RPM.

The resulting CD is nearly doing everything I want it to.

It may be worthwhile thinking about how to improve the performance by
coping more stuff into the RAM Disk. In my case there is a trade off
between the initial boot speed and the speed of the application. If it
takes too long to boot because of the copies, the users will get
annoyed.  It seems somehow better to have things actually happening on
the screen even though they are a little sluggish to start with.

NB: The kernel does a great job of making things faster by using the
free ram for caching.  This again may need some experimentation, maybe
by making the ram disk smaller the kernel can cache more and the overall
performance may be better.

PS: I am using VMWare for prototyping the CD and would highly recommend
it. As soon as the iso image has been written I can boot it in a virtual
machine.
-- 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Glen and Rosanne Eustace,
GodZone Internet Services, a division of AGRE Enterprises Ltd.,
P.O. Box 8020, Palmerston North, New Zealand 5301
Ph/Fax: [phone], Mob: [phone], Web: www.godzone.net.nz

"A Ministry specialising in providing low-cost professional Internet
Services to NZ Christian Churches, Ministries and Organisations"

[jeff]

Glen Eustace wrote:
>  It may be worthwhile thinking about how to improve the performance
> by
>  coping more stuff into the RAM Disk. In my case there is a trade off
>  between the initial boot speed and the speed of the application. If it
>  takes too long to boot because of the copies, the users will get
>  annoyed.  It seems somehow better to have things actually happening on
>  the screen even though they are a little sluggish to start with. 
For performance a few things have come to mind.

1) What about running the prelinker in the chroot before the ISO is rolled?

2) Currently mkzftree is using -z 3 for compression. What is the 
tradeoff between -z 9 and -z 3? Certainly it will take longer to 
compress, but what about decompression--is it much of a CPU hit? It 
seems the bottleneck is reading from the CD--it's not CPU bound doing 
decompression (unless on a really old box). So if the files were smaller 
there would be less data to read in and hence the CD would run faster.

3) Also, since mkzftree is one of the most time consuming processes, it 
would be nice if you could pass a -p option to it (parallelize).

4) By listening to the booting of the CD, it sounds like the head of the 
CD drive seeks quite a bit. Is there any optimization we can do so the 
CD can be read (more) sequentially?

===============


It would be nice if there was an option to kadischi to either keep or 
remove the built dirs (e.g. -k == keep build tree). It's nice to poke 
around in those.

What's up with setting the video resolution? I haven't investigated this 
yet, but I've been getting a very low resolution (640x480 or maybe 
800x600). How can kudzu set a higher resolution? Can this be passed at 
the boot: prompt?

In livecd-mkinitrd.sh it makes symlinks for busybox applications. It 
would be handy if it symlinked for all the programs busybox is compiled 
with. It won't take more space, but make the initrd more "featureful" at 
zero cost.

I did a chroot into the build dir and something as simple as "adduser" 
didn't work. It returned without error, but the user wasn't added. I 
didn't look into why yet, but anyone have an idea?

Perhaps updatedb could be run, if slocate is installed.

Also, re: ash/sash from a few days ago, I haven't fully tested it, but 
since ash is just a busybox symlink I don't think ash/sash needs to be 
on the CD (which is what I initially thought).

Thanks again!

-Jeff

P.S. I'll 'zilla some of the above if they don't get shot down here.

[Glen Eustace]

On Thu, 2005-09-01 at 14:03 -0600, jeff wrote:
>  2) Currently mkzftree is using -z 3 for compression. What is the 
>  tradeoff between -z 9 and -z 3? Certainly it will take longer to 
>  compress, but what about decompression--is it much of a CPU hit? It 
>  seems the bottleneck is reading from the CD--it's not CPU bound doing 
>  decompression (unless on a really old box). So if the files were smaller 
>  there would be less data to read in and hence the CD would run faster. 
Is squashfs going to be a better solution ?
My first venture into LiveCDs was using the YETAA stuff (ADIOS). They
have used squashfs and it seemed to work quite well.  It did mean having
to add it to the kernel. Anyone know what the plans are for having
squashfs included in the std kernel ?

>  It would be nice if there was an option to kadischi to either keep or
>
>  remove the built dirs (e.g. -k == keep build tree). It's nice to poke 
>  around in those. 
I second this :-)

I wonder whether 'breaking' kadischi up into phases, similiar to
rpmbuild would be a useful model
e.g. -ba -bb, -bi, -bp

All, Burn, ISO, Prep

One can then do as much or as little of the process as you need/want.

Glen.

[Rahul Sundaram]

Glen Eustace wrote:

> On Thu, 2005-09-01 at 14:03 -0600, jeff wrote:
>   
>
> >2) Currently mkzftree is using -z 3 for compression. What is the 
> >tradeoff between -z 9 and -z 3? Certainly it will take longer to 
> >compress, but what about decompression--is it much of a CPU hit? It 
> >seems the bottleneck is reading from the CD--it's not CPU bound doing 
> >decompression (unless on a really old box). So if the files were smaller 
> >there would be less data to read in and hence the CD would run faster.
> >    
> >
>
> Is squashfs going to be a better solution ?
> My first venture into LiveCDs was using the YETAA stuff (ADIOS). They
> have used squashfs and it seemed to work quite well.  It did mean having
> to add it to the kernel. Anyone know what the plans are for having
> squashfs included in the std kernel ?
>  
Squashfs was submitted to LKML recently and there was some discussions 
surrouding that.  It is a likely candidate for the upstream kernel and 
is under use by several other Live CD solutions

regards
Rahul

[Albert Strasheim]

Hello all

On Thu, 01 Sep 2005, jeff wrote:

>  Glen Eustace wrote:
>  >It may be worthwhile thinking about how to improve the performance by
>  >coping more stuff into the RAM Disk. In my case there is a trade off
>  >between the initial boot speed and the speed of the application. If it
>  >takes too long to boot because of the copies, the users will get
>  >annoyed.  It seems somehow better to have things actually happening on
>  >the screen even though they are a little sluggish to start with.
>  
>  For performance a few things have come to mind.
>  
>  1) What about running the prelinker in the chroot before the ISO is rolled? 
Cool idea.

>  <snip> 
>  4) By listening to the booting of the CD, it sounds like the head of
> the 
>  CD drive seeks quite a bit. Is there any optimization we can do so the 
>  CD can be read (more) sequentially? 
mkisofs and SquashFS both support the idea of prioritising files to 
indicate where they should be placed on the image (see the -sort option 
in the mkisofs man page).

This way, you can determine whether files are added to the start of the 
image, i.e. the middle of the CD, where reading them is slow, or at the 
end of the image, i.e. the edge of the image, where reading is faster.

If we can find a way of instrumenting the boot process to get a list of 
files used during boot, a priority list can be built automatically. The 
boot files can probably be followed by everything in the bin and lib 
directories and then the remaining biggest files in the installation.

Cheers

Albert