Change in vdsm[master]: logging: setup log filter correctly

Zhou Zheng Sheng has uploaded a new change for review.

Change subject: logging: setup log filter correctly
......................................................................

logging: setup log filter correctly

logging.config.fileConfig() does not setup filters, it just setups
loggers, handlers and formatters. dictConfig() can setup filters but the
logging configuration is not very readable in a Python dict.

This patch subclass the log handler to be filtered, in the __init__()
method, add the filter need. Then in the logger.conf, use the subclass
instead of the original handler.

Change-Id: I76c2f77c95dd44853f070c8f7b7a8b71cfd6866c
Signed-off-by: Zhou Zheng Sheng 
---
M vdsm/logUtils.py
M vdsm/logger.conf.in
2 files changed, 16 insertions(+), 2 deletions(-)

git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/88/9688/1

diff --git a/vdsm/logUtils.py b/vdsm/logUtils.py
index 933719d..8202bde 100644
--- a/vdsm/logUtils.py
+++ b/vdsm/logUtils.py
@@ -21,6 +21,7 @@
 import logging
 import sys
 from functools import wraps
+from logging.handlers import WatchedFileHandler
 from inspect import ismethod

@@ -160,3 +161,17 @@
             raise
         except:
             self.handleError(record)
+
+
+def getFilteredClass(logHandlerClass, *filters):
+    class Filtered(logHandlerClass):
+        def __init__(self, *args, **kwargs):
+            logHandlerClass.__init__(self, *args, **kwargs)
+            for aFilter in filters:
+                self.addFilter(aFilter)
+
+    return Filtered
+
+
+FilteredWatchedFileHandler = getFilteredClass(
+    WatchedFileHandler, TracebackRepeatFilter())
diff --git a/vdsm/logger.conf.in b/vdsm/logger.conf.in
index edd4616..cd42154 100644
--- a/vdsm/logger.conf.in
+++ b/vdsm/logger.conf.in
@@ -43,9 +43,8 @@
 args=('/dev/log', handlers.SysLogHandler.LOG_USER)

[handler_logfile]
-class=logging.handlers.WatchedFileHandler
+class=logUtils.FilteredWatchedFileHandler
 args=('@VDSMLOGDIR@/vdsm.log',)
-filters=storage.misc.TracebackRepeatFilter
 level=DEBUG
 formatter=long

--
To view, visit http://gerrit.ovirt.org/9688
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I76c2f77c95dd44853f070c8f7b7a8b71cfd6866c
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Zhou Zheng Sheng

Mark Wu has posted comments on this change.

Change subject: logging: setup log filter correctly
......................................................................

Patch Set 1: Looks good to me, but someone else must approve

(1 inline comment)

Do you know any reason why python doesn't support filter in fileConfig?  Submmit a a patch for it?

....................................................
File vdsm/logUtils.py
Line 162:         except:
Line 163:             self.handleError(record)
Line 164: 
Line 165: 
Line 166: def getFilteredClass(logHandlerClass, *filters):
is it better to use getFilteredLogHandler
Line 167:     class Filtered(logHandlerClass):
Line 168:         def __init__(self, *args, **kwargs):
Line 169:             logHandlerClass.__init__(self, *args, **kwargs)
Line 170:             for aFilter in filters:

--
To view, visit http://gerrit.ovirt.org/9688
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I76c2f77c95dd44853f070c8f7b7a8b71cfd6866c
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Zhou Zheng Sheng 
Gerrit-Reviewer: Mark Wu

Zhou Zheng Sheng has posted comments on this change.

Change subject: logging: setup log filter correctly
......................................................................

Patch Set 1: (1 inline comment)

Thanks Mark Wu.  fileConfig() does not support filter maybe the original logging implementation limitation. Instead of improve fileConfig() with filter support, people add dictConfig() to improve several things including filter support (http://www.python.org/dev/peps/pep-0391/) .

Maybe we can firstly change logger.conf into YAML or JSON format, then use dictConfig() on the de-serialized dict.

....................................................
File vdsm/logUtils.py
Line 162:         except:
Line 163:             self.handleError(record)
Line 164: 
Line 165: 
Line 166: def getFilteredClass(logHandlerClass, *filters):
I think getFilteredClass() can be used on logger objects as well, so I use a more general name. Maybe it can be

def getFiltered(logClass, *filters):
Line 167:     class Filtered(logHandlerClass):
Line 168:         def __init__(self, *args, **kwargs):
Line 169:             logHandlerClass.__init__(self, *args, **kwargs)
Line 170:             for aFilter in filters:

--
To view, visit http://gerrit.ovirt.org/9688
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I76c2f77c95dd44853f070c8f7b7a8b71cfd6866c
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Zhou Zheng Sheng 
Gerrit-Reviewer: Mark Wu 
Gerrit-Reviewer: Zhou Zheng Sheng

[Jenkins CI RO]

oVirt Jenkins CI Server has posted comments on this change.

Change subject: logging: setup log filter correctly
......................................................................

Patch Set 2:

Build Started http://jenkins.ovirt.org/job/vdsm_unit_tests_manual_gerrit/270/ (1/2)

--
To view, visit http://gerrit.ovirt.org/9688
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I76c2f77c95dd44853f070c8f7b7a8b71cfd6866c
Gerrit-PatchSet: 2
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Zhou Zheng Sheng 
Gerrit-Reviewer: Mark Wu 
Gerrit-Reviewer: Zhou Zheng Sheng 
Gerrit-Reviewer: oVirt Jenkins CI Server

[Jenkins CI RO]

oVirt Jenkins CI Server has posted comments on this change.

Change subject: logging: setup log filter correctly
......................................................................

Patch Set 2:

Build Started http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/236/ (2/2)

--
To view, visit http://gerrit.ovirt.org/9688
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I76c2f77c95dd44853f070c8f7b7a8b71cfd6866c
Gerrit-PatchSet: 2
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Zhou Zheng Sheng 
Gerrit-Reviewer: Mark Wu 
Gerrit-Reviewer: Zhou Zheng Sheng 
Gerrit-Reviewer: oVirt Jenkins CI Server

Zhou Zheng Sheng has posted comments on this change.

Change subject: logging: setup log filter correctly
......................................................................

Patch Set 2: Verified

Patch set 2: fix the incorrect implement of TracebackRepeatFilter as well.

--
To view, visit http://gerrit.ovirt.org/9688
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I76c2f77c95dd44853f070c8f7b7a8b71cfd6866c
Gerrit-PatchSet: 2
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Zhou Zheng Sheng 
Gerrit-Reviewer: Mark Wu 
Gerrit-Reviewer: Zhou Zheng Sheng 
Gerrit-Reviewer: oVirt Jenkins CI Server

[Jenkins CI RO]

oVirt Jenkins CI Server has posted comments on this change.

Change subject: logging: setup log filter correctly
......................................................................

Patch Set 2:

Build Successful

http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/236/ : SUCCESS

http://jenkins.ovirt.org/job/vdsm_unit_tests_manual_gerrit/270/ : SUCCESS

--
To view, visit http://gerrit.ovirt.org/9688
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I76c2f77c95dd44853f070c8f7b7a8b71cfd6866c
Gerrit-PatchSet: 2
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Zhou Zheng Sheng 
Gerrit-Reviewer: Mark Wu 
Gerrit-Reviewer: Zhou Zheng Sheng 
Gerrit-Reviewer: oVirt Jenkins CI Server

[Jenkins CI RO]

oVirt Jenkins CI Server has posted comments on this change.

Change subject: logging: setup log filter correctly
......................................................................

Patch Set 3:

Build Started http://jenkins.ovirt.org/job/vdsm_unit_tests_manual_gerrit/678/ (1/2)

--
To view, visit http://gerrit.ovirt.org/9688
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I76c2f77c95dd44853f070c8f7b7a8b71cfd6866c
Gerrit-PatchSet: 3
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Zhou Zheng Sheng 
Gerrit-Reviewer: Mark Wu 
Gerrit-Reviewer: Zhou Zheng Sheng 
Gerrit-Reviewer: oVirt Jenkins CI Server

[Jenkins CI RO]

oVirt Jenkins CI Server has posted comments on this change.

Change subject: logging: setup log filter correctly
......................................................................

Patch Set 3:

Build Started http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/643/ (2/2)

--
To view, visit http://gerrit.ovirt.org/9688
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I76c2f77c95dd44853f070c8f7b7a8b71cfd6866c
Gerrit-PatchSet: 3
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Zhou Zheng Sheng 
Gerrit-Reviewer: Mark Wu 
Gerrit-Reviewer: Zhou Zheng Sheng 
Gerrit-Reviewer: oVirt Jenkins CI Server

[Jenkins CI RO]

oVirt Jenkins CI Server has posted comments on this change.

Change subject: logging: setup log filter correctly
......................................................................

Patch Set 3:

Build Successful

http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/643/ : SUCCESS

http://jenkins.ovirt.org/job/vdsm_unit_tests_manual_gerrit/678/ : SUCCESS

--
To view, visit http://gerrit.ovirt.org/9688
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I76c2f77c95dd44853f070c8f7b7a8b71cfd6866c
Gerrit-PatchSet: 3
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Zhou Zheng Sheng 
Gerrit-Reviewer: Mark Wu 
Gerrit-Reviewer: Zhou Zheng Sheng 
Gerrit-Reviewer: oVirt Jenkins CI Server

Zhou Zheng Sheng has abandoned this change.

Change subject: logging: setup log filter correctly
......................................................................

Abandoned