replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]

[Miloslav Trmač]

On Sat, Oct 13, 2012 at 1:36 AM, Lennart Poettering
<mzerqung at 0pointer.de> wrote:
> On Fri, 12.10.12 15:29, Bill Nottingham (notting at redhat.com) wrote:
>> And we've got a lot of technology going around. journald - that's
>> technology. rsyslog - that's technology. libumberlog & ceelog - that's
>> technology.
>
> THis really makes me wonder where CEE actually belongs in this. Is
> anybody using this currently? What area is this supposed to cover that
> is not already covered by the journal or rsyslog? Is there really room
> for another format besides BSD syslog and journal records?

Given that the (udp AND tcp) syslog is the primary multi-platform log
transfer protocol in the UNIX world, we need to be able to take Linux
log data, including data originally generated by applications using
the journal API, and transport it using the syslog protocol. To be
really useful, the syslog representation needs not to loose data (e.g.
only including the MESSAGE field is not good enough).

So, we need a structured representation compatible with the syslog
protocol in any case, and Lumberjack/CEE provide one.  (And as soon as
there is a structured representation compatible with syslog, it is
something non-systemd platforms, like Debian or other UNIXes, can use
as well.)

"old rsyslog" (pre-Lumberjack/CEE) doesn't cover the structured
representation requirement.
journal format and protocol don't cover the syslog protocol
compatibility requirement.

>> If people want CEE format logs, or plain text logs, maybe journald should
>> grow those as output formats.
>
> To me it appears that CEE isn't widely accepted so far (heck, not even
> properly defined as multiple different vocabularies for fields are
> floating around), and I am bit unsure where it really fits in the big
> picture. I am a bit conservative in adding output formatting for CEE if
> it isn't clear that there is a need for CEE, that it's going to stick
> around for long and we actually have people using this.

The larger vision of CEE is to have a multi-platform event dictionary
and using the same event format to represent the same kind of event
(so that e.g.  and 'user log" can be identified and parsed without
knowing what specific piece of code generated it).  I'm personally not
sure how much that is achievable, or how much of the problem space it
can cover.[1].  In any case, complying with this would require either
modification of applications, or writing a CEE-specific log message
translator; it's not something we can magically get by establishing a
representation or protocol, or by only converting the structure of the
data that currently arrives in the journal without looking at the
content.

Using the Lumberjack/CEE representation natively would probably make
the application modification/translator implementation simpler (e.g.
the current proposals rely on nesting in the structure and other
syntax that is prohibited in the journal).  But as you say, these
specifications are not finalized yet.
    Mirek

[1] http://carolina.mff.cuni.cz/~trmac/blog/2011/structured-logging/