New subject: Change in vdsm[master]: WIP: profiling: memory: port from dowser to tracemalloc

Thursday, 22 October 2015

Francesco Romani has uploaded a new change for review.

Change subject: profiling: memory: port from dowser to tracemalloc
......................................................................

profiling: memory: port from dowser to tracemalloc

** WORK IN PROGRESS **

This patch implements integration with tracemalloc
(http://pytracemalloc.readthedocs.org/), replacing dowser module.

tracemalloc is
- included in python 3.4, thus received good amount of peer review
- more comprehensive, as it plugs inside cpython
- augmented with nice UI to analyze memory profile snapshot

It, however, requires a python VM compliant with PEP 445, and
an additional extension module installed in the system

Change-Id: If00a3af13ea48bb75d40ed11e56de04d90452ff7
Signed-off-by: Francesco Romani <fromani(a)redhat.com&gt;
---
M lib/vdsm/profiling/memory.py
1 file changed, 32 insertions(+), 30 deletions(-)


  git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/24/47624/1

diff --git a/lib/vdsm/profiling/memory.py b/lib/vdsm/profiling/memory.py
index 31f833c..80a07ac 100755
--- a/lib/vdsm/profiling/memory.py
+++ b/lib/vdsm/profiling/memory.py
@@ -24,19 +24,20 @@
 """
 
 import logging
+import os
 import threading
+import time
 
+from vdsm import constants
 from vdsm.config import config
-from vdsm.utils import traceback
 
 from .errors import UsageError
 
-# Import modules lazily when profile is started
-dowser = None
-cherrypy = None
+# Import tracemalloc lazily when profile is started
+tracemalloc = None
+
 
 _lock = threading.Lock()
-_thread = None
 
 
 def start():
@@ -48,7 +49,7 @@
 def stop():
     """ Stops application memory profiling """
     if is_enabled():
-        _stop_profiling()
+        _stop_profiling(_make_snapshot_name())
 
 
 def is_enabled():
@@ -56,48 +57,49 @@
 
 
 def is_running():
-    return _thread is not None
+    return tracemalloc and tracemalloc.is_tracing()
 
 
-@traceback()
-def _memory_viewer():
-    cherrypy.tree.mount(dowser.Root())
+def snapshot(filename=None):
+    with _lock:
+        if not is_running():
+            raise UsageError('Memory profiler must be running '
+                             'to take snapshots')
+        snapshot_name = filename or _make_snapshot_name()
+        with open(snapshot_name, 'wb') as snap:
+            snap.write(tracemalloc.take_snapshot())
+        return snapshot_name
 
-    cherrypy.config.update({
-        'server.socket_host': '0.0.0.0',
-        'server.socket_port': config.getint('devel',
'memory_profile_port')})
 
-    cherrypy.engine.start()
+def _make_snapshot_name():
+    return os.path.join(constants.P_VDSM_RUN,
+                        'vdsm_memory_%s.pickle' % _make_timestamp())
+
+
+def _make_timestamp():
+    return time.strftime('%Y%m%d_%H%M%S')
 
 
 def _start_profiling():
-    global cherrypy
-    global dowser
-    global _thread
+    global tracemalloc
 
     logging.debug("Starting memory profiling")
 
-    import cherrypy
-    import dowser
+    import tracemalloc
     # this nonsense makes pyflakes happy
-    cherrypy
-    dowser
+    tracemalloc
 
     with _lock:
         if is_running():
             raise UsageError('Memory profiler is already running')
-        _thread = threading.Thread(name='memprofile',
-                                   target=_memory_viewer)
-        _thread.daemon = True
-        _thread.start()
+        tracemalloc.start()
 
 
-def _stop_profiling():
+def _stop_profiling(filename):
     global _thread
     logging.debug("Stopping memory profiling")
     with _lock:
         if is_running():
-            cherrypy.engine.exit()
-            cherrypy.engine.block()
-            _thread.join()
-            _thread = None
+            snapshot(filename)
+            tracemalloc.clear_traces()
+            tracemalloc.stop()


-- 
To view, visit https://gerrit.ovirt.org/47624
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If00a3af13ea48bb75d40ed11e56de04d90452ff7
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Francesco Romani <fromani(a)redhat.com&gt;

    

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

Change in vdsm[master]: profiling: memory: port from dowser to tracemalloc