Change in vdsm[master]: (Schema) Replace host\port combos with NetworkAdress
by smizrahi＠redhat.com
Saggi Mizrahi has uploaded a new change for review.
Change subject: (Schema) Replace host\port combos with NetworkAdress
......................................................................
(Schema) Replace host\port combos with NetworkAdress
Change-Id: Iade8796a0168561ea0902ca512a817eb2f44aecb
Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com>
---
M vdsm_api/vdsmapi-schema.json
1 file changed, 21 insertions(+), 15 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/64/6964/1
--
To view, visit http://gerrit.ovirt.org/6964
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iade8796a0168561ea0902ca512a817eb2f44aecb
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>
9 years, 4 months
Change in vdsm[master]: Related to 846307 - Simplifying clientIF._cleanOldFiles().
by ewarszaw＠redhat.com
Eduardo has uploaded a new change for review.
Change subject: Related to 846307 - Simplifying clientIF._cleanOldFiles().
......................................................................
Related to 846307 - Simplifying clientIF._cleanOldFiles().
Change-Id: Ifd63a1f4b1c64f84d5734da033a2420b97ce0ada
Bug-Id: https://bugzilla.redhat.com/show_bug.cgi?id=846307
Signed-off-by: Eduardo <ewarszaw(a)redhat.com>
---
M vdsm/clientIF.py
1 file changed, 15 insertions(+), 20 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/26/8626/1
diff --git a/vdsm/clientIF.py b/vdsm/clientIF.py
index e9ca7c9..669c5be 100644
--- a/vdsm/clientIF.py
+++ b/vdsm/clientIF.py
@@ -1,3 +1,4 @@
+import os.path
#
# Copyright 2011 Red Hat, Inc.
#
@@ -453,24 +454,18 @@
return None
def _cleanOldFiles(self):
- for f in os.listdir(constants.P_VDSM_RUN):
+ for fName in os.listdir(constants.P_VDSM_RUN):
+ if fName in ('vdsmd.pid', 'respawn.pid', 'svdsm.pid',
+ 'svdsm.sock'):
+ continue
try:
- vmId, fileType = f.split(".", 1)
- if fileType in ["guest.socket", "monitor.socket", "pid",
- "stdio.dump", "recovery"]:
- if vmId in self.vmContainer:
- continue
- if f == 'vdsmd.pid':
- continue
- if f == 'respawn.pid':
- continue
- if f == 'svdsm.pid':
- continue
- if f == 'svdsm.sock':
- continue
- else:
- continue
- self.log.debug("removing old file " + f)
- utils.rmFile(constants.P_VDSM_RUN + f)
- except:
- pass
+ vmId, fileType = fName.split('.', 1)
+ except ValueError:
+ pass # fName does not contains '.'
+ if not fileType in ("guest.socket", "monitor.socket", "pid",
+ "stdio.dump", "recovery"):
+ continue
+ if vmId in self.vmContainer:
+ continue
+ self.log.debug("Removing old file %s", fName)
+ utils.rmFile(constants.P_VDSM_RUN + fName)
--
To view, visit http://gerrit.ovirt.org/8626
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ifd63a1f4b1c64f84d5734da033a2420b97ce0ada
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com>
9 years, 4 months
Change in vdsm[master]: Extend Image.move to move and copy subchains
by Federico Simoncelli
Federico Simoncelli has uploaded a new change for review.
Change subject: Extend Image.move to move and copy subchains
......................................................................
Extend Image.move to move and copy subchains
The method Image.move was only capable of moving and copying the entire
image chain. Now it's possible to specify an (optional) volUUID to move
or copy a subchain from the base up to volUUID.
Change-Id: Icb11818f830c6f02bcb74d69f1c119efaf100cc2
---
M vdsm/storage/image.py
M vdsm/storage/sp.py
2 files changed, 33 insertions(+), 22 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/17/3217/1
--
To view, visit http://gerrit.ovirt.org/3217
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Icb11818f830c6f02bcb74d69f1c119efaf100cc2
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>
9 years, 4 months
Change in vdsm[master]: image: unify the getChainTemplate method
by Federico Simoncelli
Federico Simoncelli has uploaded a new change for review.
Change subject: image: unify the getChainTemplate method
......................................................................
image: unify the getChainTemplate method
Change-Id: I32fedf40e8c05d4ef7d26cb9172c22d7ee6e9086
Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com>
---
M vdsm/storage/image.py
1 file changed, 11 insertions(+), 16 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/68/12868/1
diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py
index 92021ff..4c6047e 100644
--- a/vdsm/storage/image.py
+++ b/vdsm/storage/image.py
@@ -260,7 +260,7 @@
"""
chain = self.getChain(sdUUID, imgUUID, volUUID)
newsize = 0
- template = chain[0].getParentVolume()
+ template = self.getChainTemplate(chain)
if template:
newsize = template.getVolumeSize()
for vol in chain:
@@ -327,27 +327,22 @@
self.log.info("sdUUID=%s imgUUID=%s chain=%s ", sdUUID, imgUUID, chain)
return chain
- def getTemplate(self, sdUUID, imgUUID):
+ def getChainTemplate(self, chain):
"""
- Return template of the image
+ Return the template of a volume chain. None if not present.
"""
- tmpl = None
- # Find all volumes of image (excluding template)
- chain = self.getChain(sdUUID, imgUUID)
- # check if the chain is build above a template, or it is a standalone
- pvol = chain[0].getParentVolume()
- if pvol:
- tmpl = pvol
- elif chain[0].isShared():
- tmpl = chain[0]
+ try:
+ templateVolume = chain[0].getParentVolume()
+ except IndexError:
+ raise se.ImageValidationError("The chain has no volumes")
- return tmpl
+ return templateVolume
def prepare(self, sdUUID, imgUUID, volUUID=None):
chain = self.getChain(sdUUID, imgUUID, volUUID)
# Adding the image template to the chain
- tmplVolume = chain[0].getParentVolume()
+ tmplVolume = self.getChainTemplate(chain)
if tmplVolume:
chain.insert(0, tmplVolume)
@@ -489,7 +484,7 @@
fakeTemplate = False
pimg = volume.BLANK_UUID # standalone chain
# check if the chain is build above a template, or it is a standalone
- pvol = srcChain[0].getParentVolume()
+ pvol = self.getChainTemplate(srcChain)
if pvol:
# find out parent volume parameters
volParams = pvol.getVolumeParams()
@@ -790,7 +785,7 @@
raise se.ImageIsNotLegalChain(imgUUID)
chain = self.getChain(sdUUID, imgUUID)
# check if the chain is build above a template, or it is a standalone
- pvol = chain[0].getParentVolume()
+ pvol = self.getChainTemplate(chain)
if pvol:
if not pvol.isLegal() or pvol.isFake():
raise se.ImageIsNotLegalChain(imgUUID)
--
To view, visit http://gerrit.ovirt.org/12868
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I32fedf40e8c05d4ef7d26cb9172c22d7ee6e9086
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>
9 years, 4 months
Change in vdsm[master]: Added a support for a guest hibernate command.
by ghammer＠redhat.com
Gal Hammer has uploaded a new change for review.
Change subject: Added a support for a guest hibernate command.
......................................................................
Added a support for a guest hibernate command.
Change-Id: I1ad6836e61e9d91ec6f46a599541f61ff12e9737
---
M vdsm/clientIF.py
M vdsm/guestIF.py
M vdsm_cli/vdsClient.py
3 files changed, 32 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/21/1121/1
--
To view, visit http://gerrit.ovirt.org/1121
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I1ad6836e61e9d91ec6f46a599541f61ff12e9737
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Gal Hammer <ghammer(a)redhat.com>
9 years, 5 months
Change in vdsm[master]: [WIP] Added gluster geo-replication support
by barumuga＠redhat.com
Hello Ayal Baron, Timothy Asir, Saggi Mizrahi, Federico Simoncelli, Dan Kenigsberg,
I'd like you to do a code review. Please visit
http://gerrit.ovirt.org/8375
to review the following change.
Change subject: [WIP] Added gluster geo-replication support
......................................................................
[WIP] Added gluster geo-replication support
Below new verbs are added
glusterValidateSshConnection
glusterSetupSshConnection
Change-Id: Ic783abd5f1b63bc5116ce4ff2a3c7be92001a387
Signed-off-by: Bala.FA <barumuga(a)redhat.com>
---
M vdsm.spec.in
M vdsm/gluster/api.py
M vdsm/gluster/exception.py
3 files changed, 137 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/75/8375/1
diff --git a/vdsm.spec.in b/vdsm.spec.in
index 22e48fb..020344f 100644
--- a/vdsm.spec.in
+++ b/vdsm.spec.in
@@ -352,6 +352,7 @@
Requires: %{name} = %{version}-%{release}
Requires: glusterfs glusterfs-server glusterfs-fuse
+Requires: python-paramiko
%description gluster
Gluster plugin enables VDSM to serve Gluster functionalities.
diff --git a/vdsm/gluster/api.py b/vdsm/gluster/api.py
index a825de2..338cb06 100644
--- a/vdsm/gluster/api.py
+++ b/vdsm/gluster/api.py
@@ -19,11 +19,24 @@
#
from functools import wraps
+import socket
+import paramiko
+import re
from vdsm.define import doneCode
import supervdsm as svdsm
+from vdsm.config import config
+from vdsm import utils
+import exception as ge
_SUCCESS = {'status': doneCode}
+_KEYFILE = config.get('vars', 'trust_store_path') + '/keys/vdsmkey.pem'
+_sshKeyGenCommandPath = utils.CommandPath("ssh-keygen",
+ "/usr/bin/ssh-keygen",
+ )
+_SSH_COPY_ID_CMD = "umask 077; test -d ~/.ssh || mkdir ~/.ssh ; " \
+ "cat >> ~/.ssh/authorized_keys && (test -x /sbin/restorecon && " \
+ "/sbin/restorecon ~/.ssh ~/.ssh/authorized_keys >/dev/null 2>&1 || true)"
def exportAsVerb(func):
@@ -43,6 +56,52 @@
class VolumeStatus():
ONLINE = 'ONLINE'
OFFLINE = 'OFFLINE'
+
+
+class HostKeyMatchException(paramiko.SSHException):
+ def __init__(self, hostname, fingerprint, expected_fingerprint):
+ err = 'Fingerprint of Host key ' \
+ '%s for server %s does not match with %s' % \
+ (fingerprint, hostname, expected_fingerprint)
+ paramiko.SSHException.__init__(self, err)
+ self.hostname = hostname
+ self.fingerprint = fingerprint
+ self.expected_fingerprint = expected_fingerprint
+
+
+class HostKeyMatchPolicy(paramiko.AutoAddPolicy):
+ def __init__(self, expected_fingerprint):
+ self.expected_fingerprint = expected_fingerprint
+
+ def missing_host_key(self, client, hostname, key):
+ s = paramiko.util.hexlify(key.get_fingerprint())
+ fingerprint = ':'.join(re.findall('..', s))
+ if fingerprint == self.expected_fingerprint.upper():
+ paramiko.AutoAddPolicy.missing_host_key(self, client, hostname,
+ key)
+ else:
+ raise HostKeyMatchException(hostname, fingerprint,
+ self.expected_fingerprint)
+
+
+class GlusterSsh(paramiko.SSHClient):
+ def __init__(self, hostname, fingerprint, port=22, username=None,
+ password=None, pkey=None, key_filename=None, timeout=None,
+ allow_agent=True, look_for_keys=True, compress=False):
+ paramiko.SSHClient.__init__(self)
+ key_file_list = [_KEYFILE]
+ if key_filename:
+ key_file_list.append(list(key_filename))
+ self.set_missing_host_key_policy(HostKeyMatchPolicy(fingerprint))
+ try:
+ paramiko.SSHClient.connect(self, hostname, port, username,
+ password, pkey, key_file_list, timeout,
+ allow_agent, look_for_keys, compress)
+ except socket.error, e:
+ err = ['%s: %s' % (hostname, e)]
+ raise ge.GlusterSshConnectionFailedException(err=err)
+ except HostKeyMatchException, e:
+ raise ge.GlusterSshHostKeyMismatchException(err=[e.err])
class GlusterApi(object):
@@ -236,6 +295,47 @@
def volumeProfileStop(self, volumeName, options=None):
self.svdsmProxy.glusterVolumeProfileStop(volumeName)
+ def _validateSshConnection(self, hostname, fingerprint, username):
+ try:
+ ssh = GlusterSsh(hostname,
+ fingerprint,
+ username=username)
+ ssh.close()
+ return True
+ except paramiko.AuthenticationException, e:
+ raise ge.GlusterSshHostKeyAuthException(err=[str(e)])
+
+ @exportAsVerb
+ def validateSshConnection(self, hostname, fingerprint, username):
+ return self._validateSshConnection(hostname, fingerprint, username)
+
+ @exportAsVerb
+ def setupSshConnection(self, hostname, fingerprint, username, password):
+ rc, out, err = utils.execCmd([_sshKeyGenCommandPath.cmd, '-y', '-f',
+ _KEYFILE])
+ if rc != 0:
+ raise ge.GlusterSshPubKeyGenerationFailedException(rc=rc, err=err)
+
+ try:
+ ssh = GlusterSsh(hostname,
+ fingerprint,
+ username=username,
+ password=password)
+ c = ssh.get_transport().open_session()
+ stdin, stdout, stderr = c.exec_command(_SSH_COPY_ID_CMD)
+ stdin.write('\n'.join(out) + '\n')
+ stdin.flush()
+ stdin.close()
+ rc = c.recv_exit_status()
+ ssh.close()
+ if rc != 0:
+ err = stderr.read().splitlines()
+ raise ge.GlusterSshSetupExecFailedException(rc=rc, err=err)
+ except paramiko.AuthenticationException, e:
+ raise ge.GlusterSshHostAuthException(err=[str(e)])
+
+ return self._validateSshConnection(hostname, fingerprint, username)
+
def getGlusterMethods(gluster):
l = []
diff --git a/vdsm/gluster/exception.py b/vdsm/gluster/exception.py
index 6d94ae3..0143a5e 100644
--- a/vdsm/gluster/exception.py
+++ b/vdsm/gluster/exception.py
@@ -377,3 +377,39 @@
class GlusterHostsListFailedException(GlusterHostException):
code = 4407
message = "Hosts list failed"
+
+
+# Ssh
+class GlusterSshException(GlusterException):
+ code = 4500
+ message = "Gluster ssh exception"
+
+
+class GlusterSshConnectionFailedException(GlusterSshException):
+ code = 4501
+ message = "SSH connection failed"
+
+
+class GlusterSshHostKeyMismatchException(GlusterSshException):
+ code = 4502
+ message = "Host key match failed"
+
+
+class GlusterSshHostKeyAuthException(GlusterSshException):
+ code = 4503
+ message = "SSH host key authentication failed"
+
+
+class GlusterSshHostAuthException(GlusterSshException):
+ code = 4504
+ message = "SSH host authentication failed"
+
+
+class GlusterSshPubKeyGenerationFailedException(GlusterSshException):
+ code = 4505
+ message = "SSH public key generation failed"
+
+
+class GlusterSshSetupExecFailedException(GlusterSshException):
+ code = 4506
+ message = "SSH key setup execution failed"
--
To view, visit http://gerrit.ovirt.org/8375
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic783abd5f1b63bc5116ce4ff2a3c7be92001a387
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Bala.FA <barumuga(a)redhat.com>
Gerrit-Reviewer: Ayal Baron <abaron(a)redhat.com>
Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com>
Gerrit-Reviewer: Federico Simoncelli <fsimonce(a)redhat.com>
Gerrit-Reviewer: Saggi Mizrahi <smizrahi(a)redhat.com>
Gerrit-Reviewer: Timothy Asir <tjeyasin(a)redhat.com>
9 years, 5 months
Change in vdsm[master]: Add qemu's memory usage to VM statistics.
by ghammer＠redhat.com
Gal Hammer has uploaded a new change for review.
Change subject: Add qemu's memory usage to VM statistics.
......................................................................
Add qemu's memory usage to VM statistics.
Change-Id: Ibeb35759454c4a9b41e1303956267e93ca3545a0
Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=799285
Signed-off-by: Gal Hammer <ghammer(a)redhat.com>
---
M vdsm/config.py.in
M vdsm/libvirtvm.py
2 files changed, 14 insertions(+), 1 deletion(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/06/9006/1
diff --git a/vdsm/config.py.in b/vdsm/config.py.in
index df85e7e..ee1627b 100644
--- a/vdsm/config.py.in
+++ b/vdsm/config.py.in
@@ -111,6 +111,8 @@
('vm_sample_net_interval', '5', None),
('vm_sample_net_window', '2', None),
+
+ ('vm_sample_memory_interval', '2', None),
('trust_store_path', '@TRUSTSTORE@',
'Where the certificates and keys are situated.'),
diff --git a/vdsm/libvirtvm.py b/vdsm/libvirtvm.py
index 86e39a3..f76f35c 100644
--- a/vdsm/libvirtvm.py
+++ b/vdsm/libvirtvm.py
@@ -91,10 +91,13 @@
self._sampleNet,
config.getint('vars', 'vm_sample_net_interval'),
config.getint('vars', 'vm_sample_net_window')))
+ self.sampleMem = (utils.AdvancedStatsFunction(self._sampleMem,
+ config.getint('vars', 'vm_sample_memory_interval')))
self.addStatsFunction(
self.highWrite, self.updateVolumes, self.sampleCpu,
- self.sampleDisk, self.sampleDiskLatency, self.sampleNet)
+ self.sampleDisk, self.sampleDiskLatency, self.sampleNet,
+ self.sampleMem)
def _highWrite(self):
if not self._vm.isDisksStatsCollectionEnabled():
@@ -168,6 +171,14 @@
netSamples[nic.name] = self._vm._dom.interfaceStats(nic.name)
return netSamples
+ def _sampleMem(self):
+ memUsage = {}
+ for line in open('/proc/%d/status' %(self.conf['pid'])):
+ var, value = line.strip().split()[0:2]
+ if var in ('VmSize:', 'VmRSS:', 'VmData:'):
+ memUsage[var[:-1]] = long(value)
+ return memUsage
+
def _diff(self, prev, curr, val):
return prev[val] - curr[val]
--
To view, visit http://gerrit.ovirt.org/9006
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ibeb35759454c4a9b41e1303956267e93ca3545a0
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Gal Hammer <ghammer(a)redhat.com>
9 years, 5 months
Change in vdsm[master]: [WIP] Start moving proc parsing to it's own module
by smizrahi＠redhat.com
Saggi Mizrahi has uploaded a new change for review.
Change subject: [WIP] Start moving proc parsing to it's own module
......................................................................
[WIP] Start moving proc parsing to it's own module
Change-Id: I7ba84c7ece95bdef7448a7c7af277e7f58695401
Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com>
---
M vdsm.spec.in
M vdsm/API.py
M vdsm/Makefile.am
M vdsm/caps.py
A vdsm/procfs.py
M vdsm/utils.py
M vdsm/vm.py
7 files changed, 53 insertions(+), 40 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/13/7513/1
diff --git a/vdsm.spec.in b/vdsm.spec.in
index bd01c2a..1f01961 100644
--- a/vdsm.spec.in
+++ b/vdsm.spec.in
@@ -573,6 +573,7 @@
%{_datadir}/%{vdsm_name}/supervdsmServer.py*
%{_datadir}/%{vdsm_name}/vmChannels.py*
%{_datadir}/%{vdsm_name}/vmContainer.py*
+%{_datadir}/%{vdsm_name}/procfs.py*
%{_datadir}/%{vdsm_name}/tc.py*
%{_datadir}/%{vdsm_name}/vdsm
%{_datadir}/%{vdsm_name}/vdsm-restore-net-config
diff --git a/vdsm/API.py b/vdsm/API.py
index 720c3b9..aab69cd 100644
--- a/vdsm/API.py
+++ b/vdsm/API.py
@@ -37,6 +37,7 @@
from vdsm.define import doneCode, errCode, Kbytes, Mbytes
import caps
from vdsm.config import config
+import procfs
import supervdsm
@@ -864,7 +865,7 @@
"""
def _readSwapTotalFree():
- meminfo = utils.readMemInfo()
+ meminfo = procfs.meminfo()
return meminfo['SwapTotal'] / 1024, meminfo['SwapFree'] / 1024
stats = {}
@@ -1111,17 +1112,16 @@
memCommitted = self._memCommitted()
resident = 0
for v in self._cif.vmContainer.getVMs():
- if v.conf['pid'] == '0':
- continue
try:
- statmfile = file('/proc/' + v.conf['pid'] + '/statm')
- resident += int(statmfile.read().split()[1])
+ resident += v.statm().resident
except:
pass
+
resident *= PAGE_SIZE_BYTES
- meminfo = utils.readMemInfo()
- freeOrCached = (meminfo['MemFree'] +
- meminfo['Cached'] + meminfo['Buffers']) * Kbytes
+
+ meminfo = procfs.meminfo()
+ freeOrCached = (meminfo['MemFree'] + meminfo['Cached'] +
+ meminfo['Buffers']) * Kbytes
return freeOrCached + resident - memCommitted - \
config.getint('vars', 'host_mem_reserve') * Mbytes
diff --git a/vdsm/Makefile.am b/vdsm/Makefile.am
index 574d762..1a3ac43 100644
--- a/vdsm/Makefile.am
+++ b/vdsm/Makefile.am
@@ -47,6 +47,7 @@
momIF.py \
neterrors.py \
parted_utils.py \
+ procfs.py \
pthread.py \
supervdsm.py \
supervdsmServer.py \
diff --git a/vdsm/caps.py b/vdsm/caps.py
index f1641ff..39fc837 100644
--- a/vdsm/caps.py
+++ b/vdsm/caps.py
@@ -41,6 +41,7 @@
from vdsm import utils
from vdsm import constants
import storage.hba
+import procfs
# For debian systems we can use python-apt if available
try:
@@ -271,7 +272,7 @@
caps['HBAInventory'] = storage.hba.HBAInventory()
caps['vmTypes'] = ['kvm']
- caps['memSize'] = str(utils.readMemInfo()['MemTotal'] / 1024)
+ caps['memSize'] = str(procfs.meminfo()['MemTotal'] / 1024)
caps['reservedMem'] = str(
config.getint('vars', 'host_mem_reserve') +
config.getint('vars', 'extra_mem_reserve'))
diff --git a/vdsm/procfs.py b/vdsm/procfs.py
new file mode 100644
index 0000000..29fc973
--- /dev/null
+++ b/vdsm/procfs.py
@@ -0,0 +1,31 @@
+from collections import namedtuple
+
+buffsize = 4096
+
+MemStat = namedtuple("MemStat",
+ "size, resident, share, text, UNUSED1, data, UNUSED2")
+
+
+def statm(pid):
+ """
+ Parses statm for a pid. Note all results are in pages.
+ """
+ with open("/proc/%d/statm" % pid, "rb") as f:
+ return MemStat(*(int(val) for val in f.read().split()))
+
+
+def meminfo():
+ """
+ Parse ``/proc/meminfo`` and return its content as a dictionary.
+
+ note.
+ All values are in KB
+ """
+ meminfo = {}
+ with open("/proc/meminfo", "rb") as f:
+ f.seek(0)
+ lines = f.readlines()
+ for var, val in (l.split()[0:2] for l in lines):
+ meminfo[var[:-1]] = int(val)
+
+ return meminfo
diff --git a/vdsm/utils.py b/vdsm/utils.py
index 5e2d4e5..048a528 100644
--- a/vdsm/utils.py
+++ b/vdsm/utils.py
@@ -19,7 +19,8 @@
#
"""
-A module containing miscellaneous functions and classes that are user plentifuly around vdsm.
+A module containing miscellaneous functions and classes that are user
+plentifuly around vdsm.
.. attribute:: utils.symbolerror
@@ -28,7 +29,8 @@
from SimpleXMLRPCServer import SimpleXMLRPCServer
import SocketServer
import threading
-import os, time
+import os
+import time
import logging
import errno
import subprocess
@@ -42,6 +44,7 @@
import constants
from config import config
import netinfo
+import procfs
_THP_STATE_PATH = '/sys/kernel/mm/transparent_hugepage/enabled'
if not os.path.exists(_THP_STATE_PATH):
@@ -63,34 +66,6 @@
os.unlink(fileToRemove)
except:
pass
-
-def readMemInfo():
- """
- Parse ``/proc/meminfo`` and return its content as a dictionary.
-
- For a reason unknown to me, ``/proc/meminfo`` is is sometime
- empty when opened. If that happens, the function retries to open it
- 3 times.
-
- :returns: a dictionary representation of ``/proc/meminfo``
- """
- # FIXME the root cause for these retries should be found and fixed
- tries = 3
- meminfo = {}
- while True:
- tries -= 1
- try:
- lines = []
- lines = file('/proc/meminfo').readlines()
- for line in lines:
- var, val = line.split()[0:2]
- meminfo[var[:-1]] = int(val)
- return meminfo
- except:
- logging.warning(lines, exc_info=True)
- if tries <= 0:
- raise
- time.sleep(0.1)
#Threaded version of SimpleXMLRPCServer
class SimpleThreadedXMLRPCServer(SocketServer.ThreadingMixIn, SimpleXMLRPCServer):
@@ -225,7 +200,7 @@
"""
BaseSample.__init__(self, pid, ifids)
self.totcpu = TotalCpuSample()
- meminfo = readMemInfo()
+ meminfo = procfs.meminfo()
freeOrCached = (meminfo['MemFree'] +
meminfo['Cached'] + meminfo['Buffers'])
self.memUsed = 100 - int(100.0 * (freeOrCached) / meminfo['MemTotal'])
diff --git a/vdsm/vm.py b/vdsm/vm.py
index c1a22b0..bd436e0 100644
--- a/vdsm/vm.py
+++ b/vdsm/vm.py
@@ -36,6 +36,7 @@
import libvirt
from vdsm import vdscli
import caps
+import procfs
DEFAULT_BRIDGE = config.get("vars", "default_bridge")
@@ -693,6 +694,9 @@
load = len(self.cif.vmContainer.getVMs())
return base * (doubler + load) / doubler
+ def statm(self):
+ return procfs.statm(int(self.conf['pid']))
+
def saveState(self):
if self.destroyed:
return
--
To view, visit http://gerrit.ovirt.org/7513
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I7ba84c7ece95bdef7448a7c7af277e7f58695401
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>
9 years, 5 months
Change in vdsm[master]: Refactor prepareVolumePath
by smizrahi＠redhat.com
Saggi Mizrahi has uploaded a new change for review.
Change subject: Refactor prepareVolumePath
......................................................................
Refactor prepareVolumePath
Change-Id: I57bb8684fd11a47843a158d13fcc2815147fa7ef
Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com>
---
M vdsm/API.py
M vdsm/clientIF.py
M vdsm/libvirtvm.py
M vdsm/storage/devicemapper.py
M vdsm/vm.py
5 files changed, 93 insertions(+), 63 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/55/7755/1
diff --git a/vdsm/API.py b/vdsm/API.py
index 720c3b9..c324d79 100644
--- a/vdsm/API.py
+++ b/vdsm/API.py
@@ -173,8 +173,7 @@
# NOTE: pickled params override command-line params. this
# might cause problems if an upgrade took place since the
# parmas were stored.
- fname = self._cif.prepareVolumePath(paramFilespec)
- try:
+ with self._cif.preparedDrive(paramFilespec) as fname:
with file(fname) as f:
pickledMachineParams = pickle.load(f)
@@ -183,8 +182,6 @@
+ str(pickledMachineParams))
self.log.debug('former conf ' + str(vmParams))
vmParams.update(pickledMachineParams)
- finally:
- self._cif.teardownVolumePath(paramFilespec)
except:
self.log.error("Error restoring VM parameters",
exc_info=True)
@@ -299,9 +296,15 @@
:param hiberVolHandle: opaque string, indicating the location of
hibernation images.
"""
- params = {'vmId': self._UUID, 'mode': 'file',
- 'hiberVolHandle': hibernationVolHandle}
- response = self.migrate(params)
+ v = self._getVmObject()
+ if v is None:
+ return errCode['noVM']
+
+ try:
+ response = self.hibernate(hibernationVolHandle)
+ except vm.WrongStateError:
+ response = errCode['noVM']
+
if not response['status']['code']:
response['status']['message'] = 'Hibernation process starting'
return response
diff --git a/vdsm/clientIF.py b/vdsm/clientIF.py
index 55a7fc9..0446eb2 100644
--- a/vdsm/clientIF.py
+++ b/vdsm/clientIF.py
@@ -25,6 +25,7 @@
from xml.dom import minidom
import uuid
import errno
+from contextlib import contextmanager
from storage.dispatcher import Dispatcher
from storage.hsm import HSM
@@ -44,6 +45,7 @@
import blkid
import supervdsm
import vmContainer
+from storage import devicemapper
try:
import gluster.api as gapi
_glusterEnabled = True
@@ -239,50 +241,74 @@
self.log.info('Error finding path for device', exc_info=True)
raise vm.VolumeError(uuid)
+ def _preparePoolImage(self, drive):
+ res = self.irs.prepareImage(
+ drive['domainID'], drive['poolID'],
+ drive['imageID'], drive['volumeID'])
+
+ if res['status']['code']:
+ raise vm.VolumeError(drive)
+
+ drive['volumeChain'] = res['chain']
+ return res['path']
+
+ def _prepareDmDevice(self, drive, vmId):
+ volPath = devicemapper.getDevicePathByGuid(drive["GUID"])
+
+ if not os.path.exists(volPath):
+ raise vm.VolumeError(drive)
+
+ res = self.irs.appropriateDevice(drive["GUID"], vmId)
+ if res['status']['code']:
+ raise vm.VolumeError(drive)
+
+ return volPath
+
+ def _prepareScsiDevice(self, drive):
+ return self._getUUIDSpecPath(drive["UUID"])
+
+ def _prepareVmPayload(self, drive, vmId):
+ '''
+ vmPayload is a key in specParams
+ 'vmPayload': {'file': {'filename': 'content'}}
+ '''
+ for key, files in drive['specParams']['vmPayload'].iteritems():
+ if key == 'file':
+ svdsm = supervdsm.getProxy()
+ if drive['device'] == 'cdrom':
+ return svdsm.mkIsoFs(vmId, files)
+ elif drive['device'] == 'floppy':
+ return svdsm.mkFloppyFs(vmId, files)
+
+ raise vm.VolumeError(drive)
+
+ def _preparePath(self, drive):
+ return drive['path']
+
+ @contextmanager
+ def perparedDrive(self, drive, vmId=None):
+ path = self.prepareVolumePath(drive, vmId)
+ try:
+ yield path
+ finally:
+ self.teardownVolumePath(drive, vmId)
+
def prepareVolumePath(self, drive, vmId=None):
if type(drive) is dict:
- # PDIV drive format
if drive['device'] == 'disk' and vm.isVdsmImage(drive):
- res = self.irs.prepareImage(
- drive['domainID'], drive['poolID'],
- drive['imageID'], drive['volumeID'])
+ volPath = self._preparePoolImage(drive)
- if res['status']['code']:
- raise vm.VolumeError(drive)
-
- volPath = res['path']
- drive['volumeChain'] = res['chain']
-
- # GUID drive format
elif "GUID" in drive:
- volPath = os.path.join("/dev/mapper", drive["GUID"])
+ volPath = self._prepareDmDevice(drive, vmId)
- if not os.path.exists(volPath):
- raise vm.VolumeError(drive)
-
- res = self.irs.appropriateDevice(drive["GUID"], vmId)
- if res['status']['code']:
- raise vm.VolumeError(drive)
-
- # UUID drive format
elif "UUID" in drive:
- volPath = self._getUUIDSpecPath(drive["UUID"])
+ volPath = self._prepareScsiDevice(drive)
elif 'specParams' in drive and 'vmPayload' in drive['specParams']:
- '''
- vmPayload is a key in specParams
- 'vmPayload': {'file': {'filename': 'content'}}
- '''
- for key, files in drive['specParams']['vmPayload'].iteritems():
- if key == 'file':
- if drive['device'] == 'cdrom':
- volPath = supervdsm.getProxy().mkIsoFs(vmId, files)
- elif drive['device'] == 'floppy':
- volPath = \
- supervdsm.getProxy().mkFloppyFs(vmId, files)
+ volPath = self._prepareVmPayload(drive, vmId)
elif "path" in drive:
- volPath = drive['path']
+ volPath = self._preparePath(drive)
else:
raise vm.VolumeError(drive)
@@ -301,17 +327,22 @@
self.log.info("prepared volume path: %s", volPath)
return volPath
- def teardownVolumePath(self, drive):
- res = {'status': doneCode}
- if type(drive) == dict:
- try:
- res = self.irs.teardownImage(drive['domainID'],
- drive['poolID'], drive['imageID'])
- except KeyError:
- #This drive is not a vdsm image (quartet)
- self.log.info("Avoiding tear down drive %s", str(drive))
+ def _teardownPoolImage(self, drive):
+ try:
+ res = self.irs.teardownImage(drive['domainID'],
+ drive['poolID'], drive['imageID'])
+ return res['status']['code']
+ except KeyError:
+ #This drive is not a vdsm image (quartet)
+ self.log.info("Avoiding tear down drive %s", str(drive))
+ return doneCode
- return res['status']['code']
+ def teardownVolumePath(self, drive):
+ if type(drive) == dict:
+ return self._teardownPoolImage(drive)
+ else:
+ # Other types don't require tear down
+ return 0
def createVm(self, vmParams):
try:
@@ -320,6 +351,7 @@
except vmContainer.VmContainerError as e:
if e.errno == errno.EEXIST:
return errCode['exist']
+
return
def waitForShutdown(self, timeout=None):
diff --git a/vdsm/libvirtvm.py b/vdsm/libvirtvm.py
index a530228..ea0d017 100644
--- a/vdsm/libvirtvm.py
+++ b/vdsm/libvirtvm.py
@@ -404,11 +404,8 @@
hooks.before_vm_hibernate(self._vm._dom.XMLDesc(0), self._vm.conf)
try:
self._vm._vmStats.pause()
- fname = self._vm.cif.prepareVolumePath(self._dst)
- try:
+ with self._vm.cif.preparedDrive(self._dst) as fname:
self._vm._dom.save(fname)
- finally:
- self._vm.cif.teardownVolumePath(self._dst)
except:
self._vm._vmStats.cont()
raise
@@ -1397,11 +1394,8 @@
elif 'restoreState' in self.conf:
hooks.before_vm_dehibernate(self.conf.pop('_srcDomXML'), self.conf)
- fname = self.cif.prepareVolumePath(self.conf['restoreState'])
- try:
+ with self.cif.preparedDrive(self.conf['restoreState']) as fname:
self._connection.restore(fname)
- finally:
- self.cif.teardownVolumePath(self.conf['restoreState'])
self._dom = NotifyingVirDomain(
self._connection.lookupByUUIDString(self.id),
diff --git a/vdsm/storage/devicemapper.py b/vdsm/storage/devicemapper.py
index a1651e0..388c1cd 100644
--- a/vdsm/storage/devicemapper.py
+++ b/vdsm/storage/devicemapper.py
@@ -46,6 +46,10 @@
(major, minor)))
+def getDevicePathByGuid(devGuid):
+ return DMPATH_FORMAT % devGuid
+
+
def getSysfsPath(devName):
if "/" in devName:
raise ValueError("devName has an illegal format. "
diff --git a/vdsm/vm.py b/vdsm/vm.py
index 3aa9f52..49193d3 100644
--- a/vdsm/vm.py
+++ b/vdsm/vm.py
@@ -210,12 +210,9 @@
if ignoreParam in self._machineParams:
del self._machineParams[ignoreParam]
- fname = self._vm.cif.prepareVolumePath(self._dstparams)
- try:
- with file(fname, "w") as f:
+ with self._vm.cif.preparedDrive(self._dstparams) as fname:
+ with file(fname, "wb") as f:
pickle.dump(self._machineParams, f)
- finally:
- self._vm.cif.teardownVolumePath(self._dstparams)
self._vm.setDownStatus(NORMAL, "SaveState succeeded")
self.status = {
--
To view, visit http://gerrit.ovirt.org/7755
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I57bb8684fd11a47843a158d13fcc2815147fa7ef
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>
9 years, 5 months
Change in vdsm[master]: [wip] hsm: remove superfluous refreshes at startup
by Federico Simoncelli
Federico Simoncelli has uploaded a new change for review.
Change subject: [wip] hsm: remove superfluous refreshes at startup
......................................................................
[wip] hsm: remove superfluous refreshes at startup
During the startup it's not mandatory to refresh the iscsi connections
(the sdcache is already stale) and the lvm module can handle the lazy
initialization.
Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=870768
Change-Id: I8386d40c644c99a52f04b6b41b392abf16e3a2a6
Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com>
---
M vdsm/storage/hsm.py
1 file changed, 0 insertions(+), 3 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/76/9276/1
diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py
index 46d1605..6a5040a 100644
--- a/vdsm/storage/hsm.py
+++ b/vdsm/storage/hsm.py
@@ -364,9 +364,6 @@
self.log.warn("Failed to clean Storage Repository.", exc_info=True)
def storageRefresh():
- lvm._lvminfo.bootstrap()
- sdCache.refreshStorage()
-
fileUtils.createdir(self.tasksDir)
# TBD: Should this be run in connectStoragePool? Should tasksDir
# exist under pool link as well (for hsm tasks)
--
To view, visit http://gerrit.ovirt.org/9276
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I8386d40c644c99a52f04b6b41b392abf16e3a2a6
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>
9 years, 5 months