F20 System Wide Change: No Default Syslog

[Lennart Poettering]

On Mon, 15.07.13 14:21, Martin Langhoff (martin.langhoff at gmail.com) wrote:

> On Mon, Jul 15, 2013 at 2:01 PM, Lennart Poettering
> <mzerqung at 0pointer.de> wrote:
> > (Just to mention this: we are neither the pioneer on the
> > no-default-syslog feature nor on no-default-sendmail... A lot of other
> 
> As a cross-distro chap, I can attest to this. Specially with sendmail.
> 
> Everytime I spot sendmail in a Fedora/RHEL/CentOS box I wonder when
> will it happen. Together with sudo-by-default (and root logins
> disabled & discouraged by default).
> 
> Back to the topic, what happens when I boot from a LiveCD to diagnose
> a borked machine? Some FAQs that come to mind that google doesn't
> answer:
> 
>  - The easy part: I run journalctl -r /path/to/rootpart/ -- what if
> /var/lib/ is a mountpoint? Can I point directly to the 'database'
> file?

You'd usually mount your journal files dir, then direct "journalctl -D
/path/to/the/journal/files/dir" to it. It will then collect all files in
that dir, interleave them and present them to you.

>  - What guarantees of completeness/consistency  can we expect in
> practice from the journal in a hard poweroff situation? What is the
> price in terms of fsync() calls?
>
We sync() all journal files to disk 5min after each write the latest
(and also mark the files as "offline" when this happens, as inverse
"dirty" flag). This timeout is just a default you can configure in with
SyncIntervalSec= in journald.conf. The value is relatively high by
default, but it's difficult to find a value that works for everybody
well, since you need to balance out power management issues against
safety guarantees.

In general, the journal file is written in a scheme where we first
append things to the end, and then only update a few pointers in
front. This scheme should be fairly safe regarding incomplete writes, as
it is hard to corrupt log lines you already successfully wrote. If you
lose something you will only lose something at the end (which is not too
different from classic syslog).

When the journal starts up and finds a journal file is not marked
"offline" it will immediately rotate the file and start a new one, in
order to make sure we never fiddle with files that have incomplete
transactions in them.

journalctl will read all files during runtime, regardless if online,
rotated or offline, and will interleave them, trying to read as much of
them as it can.

So, we try hard to make sure the files stay in a complete synced/offline
state, and when that's not possible we at least try to minimize the
chance of corruption. Furthermore when reading we make the best out of
what we have.

With the default settings, in the worst case you might lose 4:59s of
logs, but even that is very unlikely.

Of course, things are different if some other form of corruption takes
place, i.e. where the fs gets corrupted so badly due to some external
condition that the middle of the journal files is bad. We do not provide
any recovery tools for that case currently.

>  - What if the database file is corrupt? On occasion, I have read
> partially corrupted logfiles with less. Sure, there was a chunk of
> crud in the middle, but I could read past it. While the un-corrupted
> bits were far from reliable, they did provide helpful info.

With MaxFileSec= or SystemMaxFileSize= you control how much log messages
we write into a journal file before we rotate and start with the next
one. If you are afraid of "middle corruptions", then I'd recommend
lowering these for your case, so that you might lose a rotated file here
or there, but all the ones that are still fine will just work.

Lennart

-- 
Lennart Poettering - Red Hat, Inc.