systemd (Was Re: tmpfs for strategic directories)

[James Findley]

On 26/05/10 11:12, Bastien Nocera wrote:
> On Wed, 2010-05-26 at 10:01 +0100, James Findley wrote:
>> On 26/05/10 04:02, Casey Dahlin wrote:
>>> On Tue, May 25, 2010 at 05:45:07PM +0200, Lennart Poettering wrote:
>>>> On Tue, 25.05.10 10:21, Casey Dahlin (cdahlin at redhat.com) wrote:
>>>>
>>>>>
>>>>> On Mon, May 24, 2010 at 09:05:31AM -0400, Tom spot Callaway wrote:
>>>>>> On 05/23/2010 04:19 AM, Kevin Kofler wrote:
>>>>>>> Lennart Poettering wrote:
>>>>>>>> So far response from the community has been very positive, but we didn't
>>>>>>>> have a fedora-devel discussion about this yet, so we'll have to see
>>>>>>>> whether we can somehow make the broader community as enthusiastic about
>>>>>>>> the whole idea as I am. ;-)
>>>>>>>
>>>>>>> I, for one, think systemd should be the default ASAP.
>>>>>>
>>>>>> Perhaps the first time I can recall that we have agreed. ;)
>>>>>>
>>>>>> ~spot
>>>>>
>>>>> Any particular reason on either of your parts?
>>>>>
>>>>> Just about everything in systemd is either set to be in upstart (simpler
>>>>> dependency syntax, xinetd-style service activation) or was discarded by it
>>>>> years ago (cgroups are a dead end).
>>>>
>>>> Oh, is that so?
>>>>
>>>> Have you actually read the blog story I put together?
>>>>
>>>
>>> Yes, but I'm going to go read it again just to be sure.
>>>
>>>> Why do you say "cgroups are a dead end"? Sure, Scott claims that, but
>>>> uh, it's not the only place where he is simply wrong and his claims
>>>> baseless. In fact it works really well, and is one of the strong points
>>>> in systemd. I simply see no alternative for it. The points Scott raised
>>>> kinda showed that he never really played around with them.
>>>>
>>>> Please read up on cgroups, before you just dismiss technology like that
>>>> as "dead end".
>>>>
>>>
>>> I did. When upstart was about to use them. 2 years ago. We chucked them by the
>>> following LPC.
>>>
>>> The problem we've found is that cgroups are too aggressive. They don't have a
>>> notion of sessions and count too much as being part of your service, so you end
>>> up with your screen session being counted as part of gdm.
>>>
>>> This is why setsid was added to the netlink connector.
>>>
>>>>> The assumption that just because its new means its more advanced is in this
>>>>> case a bit misguided.
>>>>
>>>> Well, please read the blog story. I know it's long, but it should be an
>>>> interesting read. The whole point of the blog story is to make clear the
>>>> reason systemd exists is purely technical, and that we think our design
>>>> is simply the better one.
>>>>
>>>
>>> So you have:
>>>
>>> 1) Socket activation. Part of Upstart's roadmap. Would happen sooner if you
>>> cared to submit the patch. We don't think its good enough by itself, hence the
>>> rest of Upstart, but a socket activation subsystem that could reach as far as
>>> systemd and even work standalone in settings where systemd can work is
>>> perfectly within Upstart's scope. I'd be happy to firm up the design details
>>> with you if you wanted to contribute patches.
>>>
>>> 2) Bus activation. Missed opportunity here to actually become the launchpoint
>>> for activated services. I won't criticize that too much though, as its
>>> usefulness is largely dependent on kernelspace DBus, which I've been trying to
>>> bludgeon Marcel Holtmann into turning over to the public for a year now.
>>>
>>> 3) Cutting down on the forking by replacing some of the shell scripts... cool
>>>      3a) With C code... really?
>>>
>>
>> Yeah.  I think this is odd too.
>> The blog complains about how many awk spawns there are - but this looks
>> like a complaint that belongs in the 70's.
>>
>> It really doesn't take long to launch awk. at all.  And most of the
>> things people are asking awk to do in shell scripts are very trivial,
>> requiring very little processing.
>>
>> using a simple for i in {1..1000}; do ... loop I can launch on this
>> rubbish old laptop a thousand awk processes in 3.35 seconds.  By
>> comparison, it takes 2.24 seconds to launch a thousand C hello world
>> programs.  So C is faster.  Just about.
>
> Now run a loop, in C, which does the same thing.
>
> $ time ./foo>  /dev/null
>
> real	0m0.002s
> user	0m0.001s
> sys	0m0.001s


You're comparing the wrong thing here - I was demonstrating that it 
doesn't take noticeably longer to spawn awk than a small C app on modern 
systems.
thus using:
for i in {1..1000}; do awk 'BEGIN{print "Hello World"}' > /dev/null; done
for i in {1..1000}; do ./helloworld > /dev/null; done
we compare how long it takes to start a trivial C program 1000 times to 
a trivial awk program 1000 times.
The bash for loop overhead is present in both cases, and since we are 
only interested in the difference in speed, we can ignore it.  (I 
actually ran this comparison a number of times, and used the mean value)

You're comparing how long it takes to launch an awk program 1000 times 
to how long it takes to run 1000 iterations of a loop in C.  This is not 
an especially useful thing to do.

>
>>    But the complaint in the blog
>> is that awk is called 92 times.  By this metric that costs the user 0.3
>> seconds of CPU time.  To get rid of this horrible waste of resources we
>> are compiling all our startup scripts.... wait.  Something is wrong with
>> that picture ;)
>
> There's no compiling of startup scripts, read the blog again.

"On my system the scripts in /etc/init.d call grep at least 77 times. 
awk is called 92 times, cut 23 and sed 74. [...] Of course, that has to 
be incredibly slow"  #from the blog.

"So, let's get rid of shell scripts in the boot process" #from the blog.

I'm afraid it appears to be you that needs to re-read.  I suggest that 
you read the section entitled "Keeping the First User PID Small".

I'm sorry to say that while it was slow in the 70's, and it would be too 
slow for a kernel function, on modern hardware all this completes in 
well under a second, even on this old and rubbish laptop.  that's not 
really slow by any useful *user* definition.

>
>> Modern systems just don't take very long to spawn awk. Or sed. Or cut.
>> Or bash.  IMO this sort of tradeoff between speed and ease of use hasn't
>> been appropriate in 20 years.
>
> Not very long multiplied by cold caches, and memory allocations and
> frees, etc. The problem isn't that awk is launched, is that it's
> launched a 100 times along with grep, cut, whatever kind of sticks and
> strings you can get in shell text processing. That's hardly something a
> modern init system should be doing.
>

What.  Are you seriously suggesting that launching a 340k binary, which 
links against a grand total of 4 libs, all of which will be in FS cache 
anyway as pretty much _everything_ uses them, is an issue because of 
"cold caches" and "memory allocations"? /really/?
If saving a couple of kilobytes of memory, total, across the entire boot 
process is this much of an issue for you, I rather think there are 
easier ways to achieve that.  Or, you could note that most modern 
systems appear to be sold with between 2 and 4 gigabytes of ram, and 
that it is rather hard to see any noticeable difference.

Sorry.  I like fast boots as much as the next person - this is just a 
bad trade.  We only save ourselves (much) less than a second of boot 
time, but turn a common and simple practice into a major headache.

If anyone has a bash/awk/whatever initscript that actually takes a 
user-noticeable amount of time to run, this means that you've done 
something badly wrong - it isn't the language at fault.  I would be 
happy to help you fix this - just poke me on IRC.

-siXy