Hadoop + log4j2 = fullstop

Robert Rati rrati at redhat.com
Wed May 21 16:03:42 UTC 2014


I've been working on updating the hadoop package to the latest 2.4.0 
release and at this point I've resolved all the issues but I'm now 
blocked by the log4j2 update.  log4j2 breaks the hadoop build pretty 
severely, and it doesn't seem the log4j2 team has spent much time 
thinking about how to provide backwards compatibility to existing 
log4j1.2 users.  From my investigation of log4j2:

log4j.properties file is no longer read at all
configuration file is now in XML or JSON
configuration file name is log4j2.[xml|json|jsn]
V2 isn't backwards compatible with V1.  There's a shim for v1 api but it 
will only work for a limited set of cases, and for some cases it does 
work for it turns some operations into noop calls.

This is a pretty major change and even the compatibility layer, if it 
will work for a project, does not seem to guarantee like functionality 
and minimally will require a re-do of all log4j configuration files a 
project ships.  I'm not sure many sizable upstream projects would 
undertake/accept such a drastic change very quickly.

The list of projects currently blocked by this update are:
hadoop
hbase
oozie
hive
apache-log4j-extras
amplab-tachyon

I would be surprised if there aren't a lot more.  I understand Fedora is 
always pushing for the latest versions, but for some fundamental 
packages can there be compatibility packages introduced at the same time 
as an incompatible update?  Package maintainers of dependent packages 
will still need to touch their packages and determine if the new version 
will work for them.  Providing a compat package will also allow packages 
to update to their newer versions while not held up on trying to 
integrate changes from a compatibility breaking dependency update.

Rob


More information about the bigdata mailing list