Nir Soffer has uploaded a new change for review.
Change subject: utils: Faster version of list2cmdline
......................................................................
utils: Faster version of list2cmdline
The original version from the subprocess module show on the top of a
profile on idle vdsm, taking more then 10% of cpu time. This patch
replaces it with a much simpler and faster version, which does not try
to be correct for all inputs on all platforms, but should be good enough
for vdsm commands on Linux.
Before this change, list2cmdline is the second most expensive function
in the profile:
$ profile-stats -c vdsmd.prof
Fri May 9 20:53:16 2014 vdsmd.prof
14005829 function calls (14064289 primitive calls) in 83.586 CPU seconds
Ordered by: internal time
List reduced from 1701 to 20 due to restriction <20>
ncalls tottime percall cumtime percall filename:lineno(function)
73/103 12.781 0.175 17.255 0.168 threading.py:481(Thread.run)
3230 6.309 0.002 8.907 0.003 subprocess.py:509(list2cmdline)
Using this version, list2cmdline dropped to 91th place:
2435 0.155 0.000 0.171 0.000 utils.py:646(list2cmdline)
Normalizing the cumtime by number of calls, this change saved 8.6 cpu
seconds of 83 seconds, more then 10% saving.
Change-Id: Ibfc7819c126fe51cc068cc6f9f85b1002878385b
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M lib/vdsm/utils.py
1 file changed, 21 insertions(+), 2 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/48/27548/1
diff --git a/lib/vdsm/utils.py b/lib/vdsm/utils.py
index 59b021d..bacfb8b 100644
--- a/lib/vdsm/utils.py
+++ b/lib/vdsm/utils.py
@@ -49,7 +49,6 @@
import signal
import socket
import stat
-import subprocess
import threading
import time
import zombiereaper
@@ -644,6 +643,26 @@
self._poller.close()
+def list2cmdline(seq):
+ """
+ Convert argument list for exeCmd to string that can be used in the command
+ line to perform the same command.
+
+ The builtin Python version in the subprocess module is slow, responsible
+ for 10% or cpu usage on an idle vdsm. This version may not be correct for
+ all inputs on all platforms, but it should be ok with all commands we use
+ on vdsm on Linux.
+ """
+ parts = []
+ for arg in seq:
+ if "'" in arg:
+ arg = arg.replace("'", "\'")
+ if ' ' in arg:
+ arg = "'" + arg + "'"
+ parts.append(arg)
+ return ' '.join(parts)
+
+
def execCmd(command, sudo=False, cwd=None, data=None, raw=False, logErr=True,
printable=None, env=None, sync=True, nice=None, ioclass=None,
ioclassdata=None, setsid=False, execCmdLogger=logging.root,
@@ -680,7 +699,7 @@
if not printable:
printable = command
- cmdline = repr(subprocess.list2cmdline(printable))
+ cmdline = repr(list2cmdline(printable))
execCmdLogger.debug("%s (cwd %s)", cmdline, cwd)
p = CPopen(command, close_fds=True, cwd=cwd, env=env,
--
To view, visit
http://gerrit.ovirt.org/27548
To unsubscribe, visit
http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ibfc7819c126fe51cc068cc6f9f85b1002878385b
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>