Announcing ceelog-0.1 - comments wanted

[Miloslav Trmač]

Hello,
I have just published a 0.1 of ceelog/libceelog at https://fedorahosted.org/ceelog/ :
Tarball at https://fedorahosted.org/releases/c/e/ceelog/ceelog-0.1.tar.xz
git http://git.fedorahosted.org/cgit/ceelog.git/.  A full README is appended below.

Comments (on project name, API design, command-line tool wishlist), testing, review, contributions of any kind are very welcome - on the list, in trac, to me privately, in any way you find convenient.
    Mirek

About
=====
The ceelog project provides libceelog, a library for receiving, filtering and
searching a stream or log of CEE/Lumberjack syslog records, and an associated
command-line tool, named ceelog.

The goal is to abstract the user from the backend storage (files, some kind
of local indexed storage, a remote database) and to provide efficient log
processing tools that can be used in applications and scripts for automated
log processing.

The project's home page is at https://fedorahosted.org/ceelog/ .

To get you started
==================
The ceelog(1) tool reads the "default" event source and outputs events matching
a filter.  Currently, the "default" event source is hardcoded to
/var/log/messages.

Example filters:
* A regexp (matches the unstructured event text, or the "msg" field for
  CEE/Lumberjack structured events)
    ceelog '/DHCP/'
* A field comparison (matches a CEE/Lumberjack field)
    ceelog 'uid == 0'
    ceelog 'uid != 0'
    ceelog 'trusted!uid == 0'
    ceelog 'username ~ /^guest-/'
    ceelog 'username !~ /^guest-/'
* A combination of the above
    ceelog 'trusted!uid == 0 && username ~ /^guest-/'

See the source code in src/ceelog.c for an example of a subset of the API.

Roadmap
=======
* Add "Live log file" input source that can handle messages being appended and
  log rotation.
* Document the filter expression format.
* Document ceelog(1).
* Get as close to 100% test coverage as possible.
* Support best-effort saving/restoring the current position in a source.
* Add better support for JSON types.
* Implement MongoDB input source.
* Support searching directly in the input source (e.g. to evaluate the filter
  server-side).
* ceelog(1) improvements:
  - Input processing (e.g. output only the last N recent events, block for
    more incoming events)
  - Output formatting (e.g. only output some structured fields)
  - Statistics/table output (group matching events by one field, output counts)

Bugs
====
Please consider reporting the bug to your distribution's bug tracking system.

Otherwise, please report bugs at https://fedorahosted.org/volume_key/ .  Bug
reports with patches are especially welcome.

[Miloslav Trmač]

Hello all,
----- Original Message -----
> I have just published a 0.1 of ceelog/libceelog ...

looking at the recent CEE discussion list activity, this was probably premature, and it would be better to rename this package before too many people start relying on the cee* name.

Any opinions?  I'm considering {lib,}ljlog, or perhaps even libljlog and /usr/bin/lj.
    Mirek

[Rainer Gerhards]

> ----- Original Message -----
> > I have just published a 0.1 of ceelog/libceelog ...
>
> looking at the recent CEE discussion list activity, this was probably
> premature, and it would be better to rename this package before too
> many people start relying on the cee* name.

I share this feeling, and the same concern...
> Any opinions?  I'm considering {lib,}ljlog, or perhaps even libljlog
> and /usr/bin/lj.

I am currently using CEE/lumberjack in rsyslog doc and think about switching to either lumberjack only or "structured logging". Structured logging is probably up to the point, but jlog alos makes a lot of sense. Sorry, no clear vote from me. But it definitely makes sense that everyone uses the same terms. In that sense, lumberjack may make most sense -- and as such ljlog.

Rainer

[William Heinbockel]

My vote is to keep our development efforts separate from the CEE name
for now.
While most of it has settled, there is too much potential for flux as
the effort has departed from trying to standardize on a minimal log
format and trying shoehorn in additional requirements.

Let's aim to keep our implementation simpler and focus on an
implementation that solves our logging problems. An implementation that
deviates from an unimplemented standard is better than no implementation
at all.

On Fri, 2012-10-05 at 19:16 +0000, Rainer Gerhards wrote:
> > ----- Original Message -----
> > > I have just published a 0.1 of ceelog/libceelog ...
> >
> > looking at the recent CEE discussion list activity, this was probably
> > premature, and it would be better to rename this package before too
> > many people start relying on the cee* name.
>
> I share this feeling, and the same concern...
> > Any opinions?  I'm considering {lib,}ljlog, or perhaps even libljlog
> > and /usr/bin/lj.
>
> I am currently using CEE/lumberjack in rsyslog doc and think about switching to either lumberjack only or "structured logging". Structured logging is probably up to the point, but jlog alos makes a lot of sense. Sorry, no clear vote from me. But it definitely makes sense that everyone uses the same terms. In that sense, lumberjack may make most sense -- and as such ljlog.
>
> Rainer
> _______________________________________________
> lumberjack-developers mailing list
> [e-mail]
> https://lists.fedorahosted.org/mailman/listinfo/lumberjack-developers