/etc/rc and potential startup improvements

[Shahms King]

Some of you may remember Seth Nickell talking about a replacement for
init called SystemServices.  When I first read about it I saw a few
problems with it.  First and foremost is developer buy-in.  In order for
it to be successful, it requires that the application developers
themselves provide a services API using that (potentially distribution
or Linux specific) interface.  That can be a significant barrier to
entry for an application that aims to be portable and providing startup
services are done so differently on so many platforms (some of which
will never adopt DBUS) that, in my opinion, makes it almost a
non-starter.  Additionally, I see nothing wrong with runlevels or most
of the current rc system.  The most glaring lack in the current system
is a specification of something more meaningful than a simple numeric
order.  The other solutions I've seen usually provide service
configuration and dependency information in a single, flat, file which
has the same problems as the BSD-style init scripts.

It should be possible to start programs which have no dependency on each
other in parallel. And it should be possible to remain
backwards-compatible with projects that haven't updated their
initscripts. And it should be possible to do this without changing the
current initscripts structure or "API".  Using FC1 as a base, and
chkconfig for ideas, I'm currently working on a small python-based "rc"
replacement that should be able to manage all of that with a small
addition to the headers already present at the top of an initscript.

By adding a 'depends:' line to the top of an initscript, it should be
possible to create a backwards-compatible system (installed side-by-side
with the current bash-based one) with a few extra "smarts" that can
ensure all services with explicit dependency information have their
dependencies started before they are and all services without the
'depends:' header started after all services with a lower number. 
Installing the python 'rc.py' and helper 'rc-helper' shell script (for
initscripts that don't use action, killproc, daemon, etc.) will allow
the bash-based rc script to be used in the absence of a python binary. 
Given that the goal is to write the rc.py in such a way as to use the
current runlevel information as a "guide" to dependency information only
overriding those dependencies where explicitly specified it should be
possible to gain a small measure of parallel-startup without changing
the headers at all (any services with the same number after the 'S' have
a non-deterministic order -- well,it's lexicographic, but the 'rules' of
rc say that only the numeric value counts -- and so cannot depend on
each other and may be started in parallel).

Of course, there are certain situations that will negatively impact this
proposal. Some services sometimes require the network to be up and
sometimes don't, most databases fall into this category, X terminals,
etc.

At the moment I have it working well enough to start programs in the
specified order (whooo ;-P) essentially a python port of the rc file,
actually adding some dependency information to my initscripts is the
next step, I'm just posting to ask the question: would anyone be
interested in this purely-optional add-on if I manage to get it working
to my satisfaction?  Are there other people out there working on another
solution to the initscript problem?

-- 
Shahms King <shahms(a)shahms.com&gt;

[Bill Nottingham]

Shahms King (shahms(a)shahms.com) said: 
>  At the moment I have it working well enough to start programs in the
>  specified order (whooo ;-P) essentially a python port of the rc file,
>  actually adding some dependency information to my initscripts is the
>  next step, I'm just posting to ask the question: would anyone be
>  interested in this purely-optional add-on if I manage to get it working
>  to my satisfaction?  Are there other people out there working on another
>  solution to the initscript problem? 
Sure, various implementations of this already exist. Not to
discourage you, but you may want to look at:

http://www.fefe.de/minit/
http://smarden.org/runit/
http://john.fremlin.de/programs/linux/jinit/
http://www.atnf.csiro.au/people/rgooch/linux/boot-scripts/
http://www.fastboot.org/

Some discussion points:

- python is somewhat heavyweight. Moreover, you'd need to rearchitect
  other things to make sure /usr is there before rc starts. :)
- Dependency headers are already specified in the LSB, might as well
  use those tags
- We've done testing with some of these alternatives. It *really doesn't*
  provide that much of a speedup just by paralellizing things. At
  most we saw 10-15%.

IMO, the way to faster boot is simple, yet hard: Do Less.

Bill

[Shahms King]

On Fri, 2004-02-06 at 09:32, Bill Nottingham wrote:
>  Shahms King (shahms(a)shahms.com) said: 
>  
>  Some discussion points:
>  
>  - python is somewhat heavyweight. Moreover, you'd need to rearchitect
>    other things to make sure /usr is there before rc starts. :)
>  - Dependency headers are already specified in the LSB, might as well
>    use those tags
>  - We've done testing with some of these alternatives. It *really doesn't*
>    provide that much of a speedup just by paralellizing things. At
>    most we saw 10-15%.
>  
>  IMO, the way to faster boot is simple, yet hard: Do Less.
>  
>  Bill 
Thanks for that, by the way.  Of course, all of the systems you pointed
out suffer from the fact that they are replacing init, or require major
changes to existing initscripts.  I'm just replacing rc (I happen to
*like* SysV initscripts -- for the most part).  Additionally, the lack
of /usr was an integral concern of mine, which is why the "proposed" way
to start this is from the traditional rc:

if [ -x /usr/bin/python ]; then
  exec /usr/bin/python /etc/rc.py
fi

Again, my primary motivation is making what improvements I can while
maintaining complete backwards-compatibility for situations where the
"new-fangled" approach can't work and to ensure a deterministic startup
order for scripts without the appropriate headers.

Of course, rewriting it as a statically (or minimally) linked C program
is also possible, the python is mostly proof-of-concept.

Man, that's a lot of headers to support, fortunately I really only care
about "Required-Start" ;-)

I should have a non-parallel version finished pretty soon and I will put
it up somewhere for anyone who is interested.

If I can get 10-15% "for free" (i.e. an optional component that will be
used if available) I'd say it's worth it.

-- 
Shahms King <shahms(a)shahms.com&gt;

[Mike A. Harris]

On Fri, 6 Feb 2004, Shahms King wrote:

> If I can get 10-15% "for free" (i.e. an optional component
> that will be
> used if available) I'd say it's worth it. 
"for free" perhaps, but it isn't "for free", so not worth it.  In

order to make any _major_ change to the system like this, there 
_has_ to be _major_ gains beyond just a crappy 15%.  If it were 
say 500% faster, then it might be worth considering.


-- 
Mike A. Harris     ftp://people.redhat.com/mharris
OS Systems Engineer - XFree86 maintainer - Red Hat