On Tue, 9 Oct 2012, Gergely Nagy wrote:
Botond Botyanszki boti@nxlog.org writes:
People will just keep shipping plain JSON without any further standard enforced , or create there own. They can get more productive by ignoring all of cee/lj instead of trying to follow whether to use @cee, @lj, or @WTF.
That's what I did too.
With my application developer hat on, both CEE and LJ would've made my life far more difficult than it needs be, so I stopped caring about both.
I think there are two different issues here that are getting mixed up.
1. how to transport (and filter) structured logs
2. standardizing the format of such logs so that programs that consume the logs no longer have to have a different parser for each application.
I understand that Gergely and Botond are focused on #1, and with that, just accepting JSON and XML and being able to process them is 'good enough'
I think that #2 is a bit of a pipe dream, and I'm not sure it's even desirable to try and specify things completely enough for the approach to work.
But I do think there is room for a little bit of standardization.
Syslog has four standard fields, pri, timestamp, hostname, programname. Even that little bit of standardization has enabled a huge amount of compatibility between programs. I think we need to be careful about trying to extend this. But that being said, there is a lot of interest in being able to gather the SCM_CREDENTIALS and have those in the logs. Having this information in the logs in a way that makes it impossible for the consumer of the logs to tell if the data was created by a rogue application or was gathered by the logging program defeats the purpose of this data, so I think it's worth extending the standardization just a little bit to reserve a prefix for this sort of data and name these fields.
going much beyond that, I expect the standardization to either break down, be ignored, or go the cee route where they spend so much time trying to standardize things that it doesn't matter.
David Lang