Francesco Romani has uploaded a new change for review.
Change subject: profiling: memory: port from dowser to tracemalloc ......................................................................
profiling: memory: port from dowser to tracemalloc
** WORK IN PROGRESS **
This patch implements integration with tracemalloc (http://pytracemalloc.readthedocs.org/), replacing dowser module.
tracemalloc is - included in python 3.4, thus received good amount of peer review - more comprehensive, as it plugs inside cpython - augmented with nice UI to analyze memory profile snapshot
It, however, requires a python VM compliant with PEP 445, and an additional extension module installed in the system
Change-Id: If00a3af13ea48bb75d40ed11e56de04d90452ff7 Signed-off-by: Francesco Romani fromani@redhat.com --- M lib/vdsm/profiling/memory.py 1 file changed, 32 insertions(+), 30 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/24/47624/1
diff --git a/lib/vdsm/profiling/memory.py b/lib/vdsm/profiling/memory.py index 31f833c..80a07ac 100755 --- a/lib/vdsm/profiling/memory.py +++ b/lib/vdsm/profiling/memory.py @@ -24,19 +24,20 @@ """
import logging +import os import threading +import time
+from vdsm import constants from vdsm.config import config -from vdsm.utils import traceback
from .errors import UsageError
-# Import modules lazily when profile is started -dowser = None -cherrypy = None +# Import tracemalloc lazily when profile is started +tracemalloc = None +
_lock = threading.Lock() -_thread = None
def start(): @@ -48,7 +49,7 @@ def stop(): """ Stops application memory profiling """ if is_enabled(): - _stop_profiling() + _stop_profiling(_make_snapshot_name())
def is_enabled(): @@ -56,48 +57,49 @@
def is_running(): - return _thread is not None + return tracemalloc and tracemalloc.is_tracing()
-@traceback() -def _memory_viewer(): - cherrypy.tree.mount(dowser.Root()) +def snapshot(filename=None): + with _lock: + if not is_running(): + raise UsageError('Memory profiler must be running ' + 'to take snapshots') + snapshot_name = filename or _make_snapshot_name() + with open(snapshot_name, 'wb') as snap: + snap.write(tracemalloc.take_snapshot()) + return snapshot_name
- cherrypy.config.update({ - 'server.socket_host': '0.0.0.0', - 'server.socket_port': config.getint('devel', 'memory_profile_port')})
- cherrypy.engine.start() +def _make_snapshot_name(): + return os.path.join(constants.P_VDSM_RUN, + 'vdsm_memory_%s.pickle' % _make_timestamp()) + + +def _make_timestamp(): + return time.strftime('%Y%m%d_%H%M%S')
def _start_profiling(): - global cherrypy - global dowser - global _thread + global tracemalloc
logging.debug("Starting memory profiling")
- import cherrypy - import dowser + import tracemalloc # this nonsense makes pyflakes happy - cherrypy - dowser + tracemalloc
with _lock: if is_running(): raise UsageError('Memory profiler is already running') - _thread = threading.Thread(name='memprofile', - target=_memory_viewer) - _thread.daemon = True - _thread.start() + tracemalloc.start()
-def _stop_profiling(): +def _stop_profiling(filename): global _thread logging.debug("Stopping memory profiling") with _lock: if is_running(): - cherrypy.engine.exit() - cherrypy.engine.block() - _thread.join() - _thread = None + snapshot(filename) + tracemalloc.clear_traces() + tracemalloc.stop()
automation@ovirt.org has posted comments on this change.
Change subject: profiling: memory: port from dowser to tracemalloc ......................................................................
Patch Set 1:
* Update tracker::IGNORE, no Bug-Url found * Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' and is a valid url. * Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.5', 'ovirt-3.4', 'ovirt-3.3'])
Francesco Romani has posted comments on this change.
Change subject: profiling: memory: port from dowser to tracemalloc ......................................................................
Patch Set 1: Code-Review-1 Verified+1
verified running the profiler.
automation@ovirt.org has posted comments on this change.
Change subject: WIP: profiling: memory: port from dowser to tracemalloc ......................................................................
Patch Set 2:
* Update tracker::IGNORE, no Bug-Url found * Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' and is a valid url. * Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.5', 'ovirt-3.4', 'ovirt-3.3'])
automation@ovirt.org has posted comments on this change.
Change subject: WIP: profiling: memory: port from dowser to tracemalloc ......................................................................
Patch Set 3:
* Update tracker: IGNORE, no Bug-Url found * Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' and is a valid url. * Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.5', 'ovirt-3.4', 'ovirt-3.3'])
automation@ovirt.org has posted comments on this change.
Change subject: WIP: profiling: memory: port from dowser to tracemalloc ......................................................................
Patch Set 4:
* Update tracker: IGNORE, no Bug-Url found * Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' and is a valid url. * Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.5', 'ovirt-3.4', 'ovirt-3.3'])
Francesco Romani has restored this change.
Change subject: WIP: profiling: memory: port from dowser to tracemalloc ......................................................................
Restored
gerrit-hooks has posted comments on this change.
Change subject: WIP: profiling: memory: port from dowser to tracemalloc ......................................................................
Patch Set 5:
* Update tracker: IGNORE, no Bug-Url found * Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' and is a valid url. * Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.6', 'ovirt-4.0'])
vdsm-patches@lists.fedorahosted.org