vdsm-patches August 2012

vdsm-patches@lists.fedorahosted.org

61 participants
390 discussions

Change in vdsm[master]: More RHEV -> oVirt conversion
by Ryan Harper 02 Jan '13

02 Jan '13

Ryan Harper has uploaded a new change for review. Change subject: More RHEV -> oVirt conversion ...................................................................... More RHEV -> oVirt conversion Change-Id: I5f54b4daca62f560023e8535dc578e4c62edc4a1 Signed-off-by: Ryan Harper <ryanh(a)us.ibm.com> --- M vdsm/vdsmd.init.in 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/90/3290/1 -- To view, visit http://gerrit.ovirt.org/3290 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I5f54b4daca62f560023e8535dc578e4c62edc4a1 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Ryan Harper <ryanh(a)us.ibm.com>

5 10

Change in vdsm[master]: Change RHEV to oVirt in guestIF.py
by Ryan Harper 02 Jan '13

02 Jan '13

Ryan Harper has uploaded a new change for review. Change subject: Change RHEV to oVirt in guestIF.py ...................................................................... Change RHEV to oVirt in guestIF.py Change-Id: I238e8451ea4a402713b6216e1231f670132742a0 Signed-off-by: Ryan Harper <ryanh(a)us.ibm.com> --- M vdsm/guestIF.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/89/3289/1 -- To view, visit http://gerrit.ovirt.org/3289 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I238e8451ea4a402713b6216e1231f670132742a0 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Ryan Harper <ryanh(a)us.ibm.com>

4 4

Change in vdsm[master]: Remove rhevm from vds_bootstrap/
by Ryan Harper 02 Jan '13

02 Jan '13

Ryan Harper has uploaded a new change for review. Change subject: Remove rhevm from vds_bootstrap/ ...................................................................... Remove rhevm from vds_bootstrap/ sed -e 's,RHEVM,OVIRT,g' -e 's,RHEV,oVirt,g' Change-Id: Ib28617f0fa45ff867fcd85eeb7bd49d8d5220d4d Signed-off-by: Ryan Harper <ryanh(a)us.ibm.com> --- M vds_bootstrap/vds_bootstrap.py M vds_bootstrap/vds_bootstrap_complete.py 2 files changed, 6 insertions(+), 6 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/87/3287/1 -- To view, visit http://gerrit.ovirt.org/3287 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ib28617f0fa45ff867fcd85eeb7bd49d8d5220d4d Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Ryan Harper <ryanh(a)us.ibm.com>

3 7

Change in vdsm[master]: Change vmchannel name to remove rhevm
by Ryan Harper 02 Jan '13

02 Jan '13

Ryan Harper has uploaded a new change for review. Change subject: Change vmchannel name to remove rhevm ...................................................................... Change vmchannel name to remove rhevm Change-Id: I95b44512052bdb5fed8c8c1486eafc41b85b5f13 Signed-off-by: Ryan Harper <ryanh(a)us.ibm.com> --- M vdsm/libvirtvm.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/88/3288/1 -- To view, visit http://gerrit.ovirt.org/3288 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I95b44512052bdb5fed8c8c1486eafc41b85b5f13 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Ryan Harper <ryanh(a)us.ibm.com>

3 4

Change in vdsm[master]: Change vdsmrepo to something non-rhev specific
by Ryan Harper 02 Jan '13

02 Jan '13

Ryan Harper has uploaded a new change for review. Change subject: Change vdsmrepo to something non-rhev specific ...................................................................... Change vdsmrepo to something non-rhev specific Instead of /rhevm use something more distro agnostic for storage repo path. Model this after libvirt (/var/lib/libvirt/images). Change-Id: I080a161f6338d341c986e0aaa2270a90be10891f Signed-off-by: Ryan Harper <ryanh(a)us.ibm.com> --- M configure.ac M vdsm.spec.in 2 files changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/86/3286/1 -- To view, visit http://gerrit.ovirt.org/3286 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I080a161f6338d341c986e0aaa2270a90be10891f Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Ryan Harper <ryanh(a)us.ibm.com>

6 14

Change in vdsm[master]: Fix PEP8 violations in vm.py
by smizrahi＠redhat.com 27 Dec '12

27 Dec '12

Saggi Mizrahi has uploaded a new change for review. Change subject: Fix PEP8 violations in vm.py ...................................................................... Fix PEP8 violations in vm.py Change-Id: I369a1dbfcf49cf1a8b1ac61fe88dd2fe3ea93b8e Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M Makefile.am M vdsm/vm.py 2 files changed, 118 insertions(+), 69 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/49/7449/1 diff --git a/Makefile.am b/Makefile.am index 611fec4..7a07608c 100644 --- a/Makefile.am +++ b/Makefile.am @@ -90,6 +90,7 @@ vdsm/storage/volume.py \ vdsm/supervdsm.py \ vdsm/supervdsmServer.py \ + vdsm/vm.py \ vdsm_cli \ vdsm_hooks/fileinject/before_vm_start.py \ vdsm_hooks/promisc/after_vm_start.py \ diff --git a/vdsm/vm.py b/vdsm/vm.py index 949a3ee..549eb8f 100644 --- a/vdsm/vm.py +++ b/vdsm/vm.py @@ -20,7 +20,9 @@ import os import time -import threading, logging +import threading +import logging +import math from vdsm import constants import tempfile import pickle @@ -50,6 +52,7 @@ A module containing classes needed for VM communication. """ + def isVdsmImage(drive): return all(k in drive.keys() for k in ('volumeID', 'domainID', 'imageID', 'poolID')) @@ -67,13 +70,17 @@ if not a.startswith('__')] return " ".join(attrs) -class _MigrationError(RuntimeError): pass + +class _MigrationError(RuntimeError): + pass + class MigrationSourceThread(threading.Thread): """ A thread that takes care of migration on the source vdsm. """ _ongoingMigrations = threading.BoundedSemaphore(1) + @classmethod def setMaxOutgoingMigrations(cls, n): """Set the initial value of the _ongoingMigrations semaphore. @@ -81,8 +88,8 @@ must not be called after any vm has been run.""" cls._ongoingMigrations = threading.BoundedSemaphore(n) - def __init__ (self, vm, dst='', dstparams='', - mode='remote', method='online', **kwargs): + def __init__(self, vm, dst='', dstparams='', mode='remote', + method='online', **kwargs): self.log = vm.log self._vm = vm self._dst = dst @@ -92,19 +99,23 @@ self._machineParams = {} self._downtime = kwargs.get('downtime') or \ config.get('vars', 'migration_downtime') - self.status = {'status': {'code': 0, 'message': 'Migration in process'}, 'progress': 0} + self.status = {'status': { + 'code': 0, + 'message': 'Migration in process'}, 'progress': 0 + } threading.Thread.__init__(self) self._preparingMigrationEvt = False self._migrationCanceledEvt = False - def getStat (self): + def getStat(self): """ Get the status of the migration. """ return self.status def _setupVdsConnection(self): - if self._mode == 'file': return + if self._mode == 'file': + return self.remoteHost = self._dst.split(':')[0] # FIXME: The port will depend on the binding being used. # This assumes xmlrpc @@ -188,7 +199,11 @@ def _finishSuccessfully(self): if self._mode != 'file': self._vm.setDownStatus(NORMAL, "Migration succeeded") - self.status = {'status': {'code': 0, 'message': 'Migration done'}, 'progress': 100} + self.status = { + 'status': { + 'code': 0, + 'message': 'Migration done'}, + 'progress': 100} else: # don't pickle transient params for ignoreParam in ('displayIp', 'display', 'pid'): @@ -203,7 +218,11 @@ self._vm.cif.teardownVolumePath(self._dstparams) self._vm.setDownStatus(NORMAL, "SaveState succeeded") - self.status = {'status': {'code': 0, 'message': 'SaveState done'}, 'progress': 100} + self.status = { + 'status': { + 'code': 0, + 'message': 'SaveState done'}, + 'progress': 100} def run(self): try: @@ -240,12 +259,15 @@ def __str__(self): return "Bad volume specification " + RuntimeError.__str__(self) -class DoubleDownError(RuntimeError): pass + +class DoubleDownError(RuntimeError): + pass VALID_STATES = ('Down', 'Migration Destination', 'Migration Source', 'Paused', 'Powering down', 'RebootInProgress', 'Restoring state', 'Saving State', 'Up', 'WaitForLaunch') + class Vm(object): """ @@ -271,7 +293,7 @@ self.conf = {'pid': '0'} self.conf.update(params) self.cif = cif - self.log = SimpleLogAdapter(self.log, {"vmId" : self.conf['vmId']}) + self.log = SimpleLogAdapter(self.log, {"vmId": self.conf['vmId']}) self.destroyed = False self._recoveryFile = constants.P_VDSM_RUN + str( self.conf['vmId']) + '.recovery' @@ -304,7 +326,7 @@ self._startTime = time.time() - float( self.conf.pop('elapsedTimeOffset', 0)) - self._usedIndices = {} #{'ide': [], 'virtio' = []} + self._usedIndices = {} # {'ide': [], 'virtio' = []} self._volumesPrepared = False self._pathsPreparedEvent = threading.Event() self._devices = {DISK_DEVICES: [], NIC_DEVICES: [], @@ -312,9 +334,15 @@ CONTROLLER_DEVICES: [], GENERAL_DEVICES: [], BALLOON_DEVICES: [], REDIR_DEVICES: []} + self.volumeUtilizationChunk = \ + config.getint('irs', 'volume_utilization_chunk_mb') + self.systemShutdownTimeout = \ + config.getint('vars', 'sys_shutdown_timeout') + def _get_lastStatus(self): SHOW_PAUSED_STATES = ('Powering down', 'RebootInProgress', 'Up') - if not self._guestCpuRunning and self._lastStatus in SHOW_PAUSED_STATES: + if (not self._guestCpuRunning and + self._lastStatus in SHOW_PAUSED_STATES): return 'Paused' return self._lastStatus @@ -344,7 +372,7 @@ def _normalizeVdsmImg(self, drv): drv['needExtend'] = False drv['reqsize'] = drv.get('reqsize', '0') - if not drv.has_key('device'): + if not 'device' in drv: drv['device'] = 'disk' if drv['device'] == 'disk': @@ -358,22 +386,26 @@ drv['apparentsize'] = 0 def __legacyDrives(self): - """ - Backward compatibility for qa scripts that specify direct paths. - """ - legacies = [] - for index, linuxName in ((0, 'hda'), (1, 'hdb'), (2, 'hdc'), (3, 'hdd')): - path = self.conf.get(linuxName) - if path: - legacies.append({'type': DISK_DEVICES, 'device': 'disk', - 'path': path, 'iface': 'ide', 'index': index, - 'truesize': 0}) - return legacies + """ + Backward compatibility for qa scripts that specify direct paths. + """ + legacies = [] + for index, linuxName in enumerate(('hda', 'hdb', 'hdc', 'hdd')): + path = self.conf.get(linuxName) + if path: + legacies.append({'type': DISK_DEVICES, 'device': 'disk', + 'path': path, 'iface': 'ide', 'index': index, + 'truesize': 0}) + return legacies def __removableDrives(self): - removables = [{'type': DISK_DEVICES, 'device': 'cdrom', 'iface': 'ide', - 'path': self.conf.get('cdrom', ''), 'index': 2, - 'truesize': 0}] + removables = [{ + 'type': DISK_DEVICES, + 'device': 'cdrom', + 'iface': 'ide', + 'path': self.conf.get('cdrom', ''), + 'index': 2, + 'truesize': 0}] floppyPath = self.conf.get('floppy') if floppyPath: removables.append({'type': DISK_DEVICES, 'device': 'floppy', @@ -440,14 +472,16 @@ """ controllers = [] # For now we create by default only 'virtio-serial' controller - controllers.append({'type': CONTROLLER_DEVICES, 'device': 'virtio-serial'}) + controllers.append({ + 'type': CONTROLLER_DEVICES, + 'device': 'virtio-serial'}) return controllers def getConfVideo(self): """ Normalize video device provided by conf. """ - vcards =[] + vcards = [] if self.conf.get('display') == 'vnc': devType = 'cirrus' elif self.conf.get('display') == 'qxl': @@ -480,9 +514,12 @@ macs = self.conf.get('macAddr', '').split(',') models = self.conf.get('nicModel', '').split(',') bridges = self.conf.get('bridge', DEFAULT_BRIDGE).split(',') - if macs == ['']: macs = [] - if models == ['']: models = [] - if bridges == ['']: bridges = [] + if macs == ['']: + macs = [] + if models == ['']: + models = [] + if bridges == ['']: + bridges = [] if len(models) < len(macs) or len(models) < len(bridges): raise ValueError('Bad nic specification') if models and not (macs or bridges): @@ -495,17 +532,20 @@ for mac, model, bridge in zip(macs, models, bridges): if model == 'pv': model = 'virtio' - nics.append({'type': NIC_DEVICES, 'macAddr': mac, 'nicModel': model, - 'network': bridge, 'device': 'bridge'}) + nics.append({ + 'type': NIC_DEVICES, + 'macAddr': mac, + 'nicModel': model, + 'network': bridge, + 'device': 'bridge'}) return nics def getConfDrives(self): """ Normalize drives provided by conf. """ - # FIXME - # Will be better to change the self.conf but this implies an API change. - # Remove this when the API parameters will be consistent. + # FIXME: Will be better to change the self.conf but this implies an API + # change. Remove this when the API parameters will be consistent. confDrives = self.conf['drives'] if self.conf.get('drives') else [] if not confDrives: confDrives.extend(self.__legacyDrives()) @@ -536,7 +576,7 @@ drives = [(order, drv) for order, drv in enumerate(confDrives)] indexed = [] for order, drv in drives: - if not self._usedIndices.has_key(drv['iface']): + if drv['iface'] not in self._usedIndices: self._usedIndices[drv['iface']] = [] idx = drv.get('index') if idx is not None: @@ -556,8 +596,8 @@ """ Reserve the required memory for this VM. """ - self.memCommitted = 2**20 * (int(self.conf['memSize']) + - config.getint('vars', 'guest_ram_overhead')) + self.memCommitted = ((2 ** 20) * (int(self.conf['memSize'])) + + config.getint('vars', 'guest_ram_overhead')) def _startUnderlyingVm(self): self.log.debug("Start") @@ -653,7 +693,7 @@ load = len(self.cif.vmContainer) return base * (doubler + load) / doubler - def saveState (self): + def saveState(self): if self.destroyed: return toSave = deepcopy(self.status()) @@ -666,7 +706,8 @@ toSave['guestIPs'] = "" if 'sysprepInf' in toSave: del toSave['sysprepInf'] - if 'floppy' in toSave: del toSave['floppy'] + if 'floppy' in toSave: + del toSave['floppy'] for drive in toSave.get('drives', []): for d in self._devices[DISK_DEVICES]: if d.isVdsmImage() and drive.get('volumeID') == d.volumeID: @@ -675,11 +716,11 @@ with tempfile.NamedTemporaryFile(dir=constants.P_VDSM_RUN, delete=False) as f: - pickle.dump(toSave, f) + pickle.dump(toSave, f) os.rename(f.name, self._recoveryFile) - def onReboot (self, withRelaunch): + def onReboot(self, withRelaunch): try: self.log.debug('reboot event') self._startTime = time.time() @@ -695,7 +736,7 @@ except: self.log.error("Reboot event failed", exc_info=True) - def onShutdown (self): + def onShutdown(self): self.log.debug('onShutdown() event') self.user_destroy = True @@ -719,12 +760,13 @@ def _lvExtend(self, block_dev, newsize=None): volID = None for d in self._devices[DISK_DEVICES]: - if not d.blockDev: continue - if d.name != block_dev: continue + if not d.blockDev: + continue + if d.name != block_dev: + continue if newsize is None: - newsize = config.getint('irs', - 'volume_utilization_chunk_mb') + (d.apparentsize + 2**20 - - 1) / 2**20 + apperantSizeMB = int(math.ceil(d.apparentsize / (2.0 ** 20.0))) + newsize = self.volumeUtilizationChunk + apperantSizeMB # TODO cap newsize by max volume size volDict = {'poolID': d.poolID, 'domainID': d.domainID, 'imageID': d.imageID, 'volumeID': d.volumeID, @@ -732,11 +774,11 @@ d.needExtend = True d.reqsize = newsize # sendExtendMsg expects size in bytes - self.cif.irs.sendExtendMsg(d.poolID, volDict, newsize * 2**20, + self.cif.irs.sendExtendMsg(d.poolID, volDict, newsize * (2 ** 20), self._afterLvExtend) self.log.debug('%s/%s (%s): apparentsize %s req %s', d.domainID, - d.volumeID, d.name, d.apparentsize / constants.MEGAB, - newsize) #in MiB + d.volumeID, d.name, d.apparentsize / constants.MEGAB, + newsize) # in MiB volID = d.volumeID break @@ -778,8 +820,8 @@ apparentsize = int(res['apparentsize']) truesize = int(res['truesize']) self.log.debug('_afterLvExtend apparentsize %s req size %s', - apparentsize / constants.MEGAB, d.reqsize) # MiB - if apparentsize >= d.reqsize * constants.MEGAB: #in Bytes + apparentsize / constants.MEGAB, d.reqsize) # MiB + if apparentsize >= d.reqsize * constants.MEGAB: # in Bytes d.needExtend = False try: self.cont() @@ -821,8 +863,8 @@ self._acquireCpuLockWithTimeout() try: if self.lastStatus in ('Migration Source', 'Saving State', 'Down'): - self.log.error('cannot cont while %s', self.lastStatus) - return errCode['unexpected'] + self.log.error('cannot cont while %s', self.lastStatus) + return errCode['unexpected'] self._underlyingCont() if hasattr(self, 'updateGuestCpuRunning'): self.updateGuestCpuRunning() @@ -860,7 +902,7 @@ self._guestEvent = 'Powering down' self.log.debug('guestAgent shutdown called') self.guestAgent.desktopShutdown(timeout, message) - agent_timeout = int(timeout) + config.getint('vars', 'sys_shutdown_timeout') + agent_timeout = int(timeout) + self.systemShutdownTimeout timer = threading.Timer(agent_timeout, self._timedShutdown) timer.start() elif utils.tobool(self.conf.get('acpiEnable', 'true')): @@ -869,8 +911,10 @@ self._acpiShutdown() # No tools, no ACPI else: - return {'status': {'code': errCode['exist']['status']['code'], - 'message': 'VM without ACPI or active SolidICE tools. Try Forced Shutdown.'}} + return {'status': { + 'code': errCode['exist']['status']['code'], + 'message': 'VM without ACPI or active SolidICE tools. ' + 'Try Forced Shutdown.'}} except: self.log.error("Shutdown failed", exc_info=True) return {'status': {'code': doneCode['code'], @@ -912,7 +956,7 @@ utils.rmFile(self._guestSocketFile) utils.rmFile(self._recoveryFile) - def setDownStatus (self, code, reason): + def setDownStatus(self, code, reason): try: self.lastStatus = 'Down' self.conf['exitCode'] = code @@ -972,7 +1016,7 @@ 'network': {}, 'disks': {}, 'monitorResponse': str(self._monitorResponse), 'nice': self._nice, - 'elapsedTime' : str(int(time.time() - self._startTime)), + 'elapsedTime': str(int(time.time() - self._startTime)), } if 'cdrom' in self.conf: stats['cdrom'] = self.conf['cdrom'] @@ -998,12 +1042,15 @@ try: stats['disks'][var] = {} for value in decStats[var]: - stats['disks'][var][value] = utils.convertToStr(decStats[var][value]) + stats['disks'][var][value] = \ + utils.convertToStr(decStats[var][value]) except: - self.log.error("Error setting vm disk stats", exc_info=True) + self.log.error("Error setting vm disk stats", + exc_info=True) - - if self.lastStatus in ('Saving State', 'Restoring state', 'Migration Source', 'Migration Destination', 'Paused'): + if self.lastStatus in ('Saving State', + 'Restoring state', 'Migration Source', + 'Migration Destination', 'Paused'): stats['status'] = self.lastStatus elif self.isMigrating(): if self._migrationSourceThread._mode == 'file': @@ -1026,8 +1073,9 @@ memUsage = 0 realMemUsage = int(stats['memUsage']) if realMemUsage != 0: - memUsage = 100 - float(realMemUsage) / int(self.conf['memSize']) * 100 - stats['memUsage'] = utils.convertToStr(int(memUsage)) + memUsage = 1 - (float(realMemUsage) / int(self.conf['memSize'])) + + stats['memUsage'] = utils.convertToStr(int(memUsage * 100)) return stats def isMigrating(self): -- To view, visit http://gerrit.ovirt.org/7449 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I369a1dbfcf49cf1a8b1ac61fe88dd2fe3ea93b8e Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

5 10

Change in vdsm[master]: Attempt to name created threads appropriately.
by peter.portante＠redhat.com 26 Dec '12

26 Dec '12

Peter Portante has uploaded a new change for review. Change subject: Attempt to name created threads appropriately. ...................................................................... Attempt to name created threads appropriately. When debugging leaked threads, or just trying to understand what threads are executing at a given point in time, it is helpful to have names for the threads that relate back to the purpose and/or individual object they are related to. Some of the threads had names, but were not directly tied back to the objects they were related to. Change-Id: I8131f4cf0ef907ad7abed24065523da9a42bb0d2 Signed-off-by: Peter Portante <peter.portante(a)redhat.com> --- M vdsm/API.py M vdsm/clientIF.py M vdsm/ksm.py M vdsm/libvirtvm.py M vdsm/storage/domainMonitor.py M vdsm/storage/hsm.py M vdsm/storage/misc.py M vdsm/storage/processPool.py M vdsm/storage/resourceManager.py M vdsm/storage/sd.py M vdsm/storage/sp.py M vdsm/storage/storageServer.py M vdsm/storage/storage_mailbox.py M vdsm/storage/sync.py M vdsm/storage/task.py M vdsm/storage/taskManager.py M vdsm/storage/threadPool.py M vdsm/supervdsmServer.py M vdsm/ut/ivdsm.py M vdsm/utils.py M vdsm/vdsmDebugPlugin.py M vdsm/vm.py 22 files changed, 95 insertions(+), 43 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/17/3517/1 -- To view, visit http://gerrit.ovirt.org/3517 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8131f4cf0ef907ad7abed24065523da9a42bb0d2 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Peter Portante <peter.portante(a)redhat.com>

3 9

Change in vdsm[master]: Move IOUSER from misc.py to constants.py
by Xu He Jie 26 Dec '12

26 Dec '12

Xu He Jie has uploaded a new change for review. Change subject: Move IOUSER from misc.py to constants.py ...................................................................... Move IOUSER from misc.py to constants.py Move IOUSER from misc.py to constants.py and add it to configure.ac. make it configurable. Change-Id: Id5d20317ac5a38d8ddd74b26272cd870aa4702b9 --- M configure.ac M vdsm/constants.py.in M vdsm/storage/misc.py M vdsm/storage/safelease.py 4 files changed, 3 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/62/3362/1 -- To view, visit http://gerrit.ovirt.org/3362 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Id5d20317ac5a38d8ddd74b26272cd870aa4702b9 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Xu He Jie <xuhj(a)linux.vnet.ibm.com>

3 4

Change in vdsm[master]: [WIP]Add text-based console support
by Xu He Jie 26 Dec '12

26 Dec '12

Xu He Jie has uploaded a new change for review. Change subject: [WIP]Add text-based console support ...................................................................... [WIP]Add text-based console support This patch use ssh protocol providing a text-based console for user. User can have another choice except spice and vnc. Text-based console consumes less bandwidth and generic ssh client can connect it. This patch try to keep same behaviour as vnc. Two new params for VM creation: consoleEnable: if true, it will enable console support for this VM consolePort: the port used by ssh server. if -1 specified, the port will be allocated automatically. The allocated port will show at getVmStats(). setVmTicket: password: used to connect this console. ttl: after expired, the console will be closed. existingConnAction: if 'disconnect' specified, the current connection will be closed, and reopen with new params. TODO: * Make console sharable. Current console only can connect by only one user * setTicket will be used by vnc and console seperately Change-Id: I69904bf7aafd4f764a256d4075c9bf71d988e7c5 Signed-off-by: Xu He Jie <xuhj(a)linux.vnet.ibm.com> --- M vdsm.spec.in M vdsm/Makefile.am A vdsm/consoleServer.py M vdsm/constants.py.in M vdsm/libvirtvm.py M vdsm/supervdsmServer.py M vdsm/utils.py M vdsm/vm.py 8 files changed, 377 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/65/7165/1 diff --git a/vdsm.spec.in b/vdsm.spec.in index bf8684a..33c8d6f 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -545,6 +545,7 @@ %{_datadir}/%{vdsm_name}/blkid.py* %{_datadir}/%{vdsm_name}/caps.py* %{_datadir}/%{vdsm_name}/clientIF.py* +%{_datadir}/%{vdsm_name}/consoleServer.py* %{_datadir}/%{vdsm_name}/API.py* %{_datadir}/%{vdsm_name}/hooking.py* %{_datadir}/%{vdsm_name}/hooks.py* diff --git a/vdsm/Makefile.am b/vdsm/Makefile.am index 80befdb..dbb4dc3 100644 --- a/vdsm/Makefile.am +++ b/vdsm/Makefile.am @@ -31,6 +31,7 @@ caps.py \ clientIF.py \ configNetwork.py \ + consoleServer.py \ debugPluginClient.py \ guestIF.py \ hooking.py \ diff --git a/vdsm/consoleServer.py b/vdsm/consoleServer.py new file mode 100755 index 0000000..172fcae --- /dev/null +++ b/vdsm/consoleServer.py @@ -0,0 +1,279 @@ +#! /usr/bin/python +# +# Copyright (C) 2012, IBM Corporation +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# + +import sys +import logging +import logging.config +import socket +import select +import os +import subprocess + +import paramiko + +from vdsm import constants +from vdsm import utils + + +class PortAllocator(): + + def __init__(self, minPort, maxPort): + self._minPort = minPort + self._maxPort = maxPort + self._bitmap = utils.Bitmap(self._maxPort - self._minPort) + self._addrs = {} + + def getPort(self): + for i in range(0, self._maxPort - self._minPort): + if self._bitmap[i] == 0x0: + servSock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + servSock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + + try: + servSock.bind(("", i + self._minPort)) + except socket.error: + continue + finally: + servSock.close() + + self._bitmap[i] = 0x1 + return i + self._minPort + + raise Exception("Can not free port on address: %s", addr) + + def freePort(self, port): + print("free", port - self._minPort) + self._bitmap[port - self._minPort] = 0x0 + +portAllocator = PortAllocator(constants.SSH_CONSOLE_MIN_PORT, + constants.SSH_CONSOLE_MAX_PORT) + + +class SSHConsole(object): + AUTO_PORT = -1 + logger = logging.getLogger("console") + + def __init__(self, pty="", addr="", port=AUTO_PORT, + passwd="", timeout=-1, hostKeyFile="", + portAlloc=portAllocator): + self._prog = None + self._pty = pty + self._addr = addr + + self._portAlloc = portAllocator + self._port = port + if self._port == self.AUTO_PORT: + self._port = self._portAlloc.getPort() + + self._passwd = passwd + self._hostKeyFile = hostKeyFile + self._timeout = timeout + + def setPty(self, pty): + self._pty = pty + + def setPasswd(self, passwd): + self._passwd = passwd + + def setTimeout(self, timeout): + self._timeout = timeout + + def getPort(self): + return self._port + + def open(self): + self.logger.info("open console with: %s", {"pty": self._pty, + "addr": self._addr, + "port": self._port, + "timeout": self._timeout, + "hostKeyFile": self._hostKeyFile}) + f = open(self._pty, "rwa+") + self._prog = subprocess.Popen([constants.EXT_PYTHON, + os.path.join(constants.P_VDSM, __file__), + self._addr, + str(self._port), + self._passwd, + str(self._timeout), + self._hostKeyFile], + stdin=f, + stdout=f, + stderr=subprocess.STDOUT) + + def close(self): + if self._prog: + self._prog.kill() + self._prog.wait() + self._portAlloc.freePort(self._port) + + def isOpen(self): + return self._prog != None and self._prog.poll() == None + + +class ParamikoServerInf(paramiko.ServerInterface): + + def __init__(self, passwd): + self._passwd = passwd + + def check_channel_request(self, kind, chanid): + if kind == 'session': + return paramiko.OPEN_SUCCEEDED + return paramiko.OPEN_FAILED_UNKNOWN_CHANNEL_TYPE + + def check_auth_password(self, username, password): + if password != self._passwd: + return paramiko.AUTH_FAILED + return paramiko.AUTH_SUCCESSFUL + + def check_channel_shell_request(self, channel): + return True + + def check_channel_pty_request(self, channel, term, width, height, + pixelwidth, pixelheight, modes): + return True + + def check_auth_none(self, username): + return paramiko.AUTH_FAILED + + +class SSHConsoleServer(object): + + def __init__(self, logger, inputf, outputf, + addr, port, passwd, timeout, hostKeyFile): + self._logger = logger + self._output = outputf + self._input = inputf + self._addr = addr + self._port = port + self._passwd = passwd + self._timeout = timeout + self._hostKeyFile = hostKeyFile + + def start(self): + try: + hostKey = paramiko.RSAKey(filename=self._hostKeyFile) + except (paramiko.SSHException, IOError) as e: + self._logger.error("Can not load key from %s", + self._hostKeyFile, + exc_info=True) + return + + try: + servSock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + servSock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + except socket.error as e: + self._logger.error("create socket failed: %s", + e.message, exc_info=True) + return + + try: + servSock.bind((self._addr, self._port)) + servSock.listen(1) + except socket.error as e: + self._logger.error("failed binding address on %s:%s", + self._addr, self._port, exc_info=True) + servSock.close() + return + + while True: + self._logger.info("ssh console server waiting for connection") + + rd, wr, er = select.select([servSock, self._input], + [], [], self._timeout) + + if len(rd) == 0: + self._logger.info("connection timeout") + break + elif self._input in rd: + # if 0 returned means input was closed + if len(self._input.read()) == 0: + self._logger.info("console closed") + break + + clientSock, addr = servSock.accept() + self._logger.info("accept connection from %s", addr) + + trans = paramiko.Transport(clientSock) + trans.add_server_key(hostKey) + serverInf = ParamikoServerInf(self._passwd) + + try: + trans.start_server(server=serverInf) + except (paramiko.SSHException, EOFError) as e: + self._logger.error("start ssh server failed: %s", e.message) + clientSock.close() + continue + + chan = trans.accept() + + if chan is None: + self._logger.error("get SSH channel failed") + continue + + while True: + try: + rd, wr, er = select.select([chan, self._output], [], []) + + if chan in rd: + buf = chan.recv(1024) + + if len(buf) == 0: + self._logger.info("ssh channel closed") + break + + os.write(self._input.fileno(), buf) + elif self._output in rd: + buf = os.read(self._output.fileno(), 1) + + if chan.send(buf) == 0: + self._logger.info("ssh channel closed") + break + else: + self._logger.error("could not be happened", + exc_info=True) + + except IOError as e: + self._logger.error("got io error", exc_info=True) + break + + self._logger.info("ssh closed") + chan.close() + clientSock.close() + trans.close() + + servSock.close() + +if __name__ == '__main__': + logging.config.fileConfig(constants.P_VDSM_CONF + 'logger.conf') + logger = logging.getLogger("consoleServer") + + try: + addr = sys.argv[1] + port = int(sys.argv[2]) + passwd = sys.argv[3] + timeout = int(sys.argv[4]) + hostKey = sys.argv[5] + + logger.info("start a ssh server") + s = SSHConsoleServer(logger, sys.stdin, sys.stdout, + addr, port, passwd, timeout, hostKey) + s.start() + except Exception as e: + logger.error("ssh server failed: %s: %s", e.message, type(e)) diff --git a/vdsm/constants.py.in b/vdsm/constants.py.in index 3a5fdce..53ae56f 100644 --- a/vdsm/constants.py.in +++ b/vdsm/constants.py.in @@ -167,3 +167,9 @@ max_fds 4096 } """ + +# +# ssh console server port constants +# +SSH_CONSOLE_MAX_PORT = 65535 +SSH_CONSOLE_MIN_PORT = 12200 diff --git a/vdsm/libvirtvm.py b/vdsm/libvirtvm.py index ddd030e..ea3b1b8 100644 --- a/vdsm/libvirtvm.py +++ b/vdsm/libvirtvm.py @@ -24,6 +24,7 @@ import time import threading import json +import os.path import vm from vdsm.define import ERROR, doneCode, errCode @@ -37,6 +38,7 @@ import caps from vdsm import netinfo import supervdsm +import consoleServer _VMCHANNEL_DEVICE_NAME = 'com.redhat.rhevm.vdsm' @@ -1232,6 +1234,7 @@ self._getUnderlyingVideoDeviceInfo() self._getUnderlyingControllerDeviceInfo() self._getUnderlyingBalloonDeviceInfo() + self._getUnderlyingConsoleDeviceInfo() # Obtain info of all unknown devices. Must be last! self._getUnderlyingUnknownDeviceInfo() @@ -1260,6 +1263,23 @@ self.guestAgent = guestIF.GuestAgent(self._guestSocketFile, self.cif.channelListener, self.log, connect=utils.tobool(self.conf.get('vmchannel', 'true'))) + + if utils.tobool(self.conf.get("enableConsole", False)): + self.log.info("vm console enabled") + + supervdsm.getProxy().changePtyOwner( + int(os.path.basename(self.conf["consolePath"]))) + + self._console = consoleServer.SSHConsole( + self.conf['consolePath'], + "", + int(self.conf.get("consolePort", -1)), + "", + -1, + os.path.join( + constants.P_VDSM_KEYS, 'vdsmkey.pem')) + + self.conf["consolePort"] = self._console.getPort() self._guestCpuRunning = self._dom.info()[0] == libvirt.VIR_DOMAIN_RUNNING if self.lastStatus not in ('Migration Destination', @@ -1931,6 +1951,21 @@ graphics.setAttribute('passwdValidTo', validto) if graphics.getAttribute('type') == 'spice': graphics.setAttribute('connected', connAct) + + if self._console: + self.log.debug("setticket for console") + if connAct == "disconnect": + if self._console.isOpen(): + self._console.close() + self._console.setPasswd(otp) + self._console.setTimeout(seconds) + self._console.open() + else: + if not self._console.isOpen(): + self._console.setPasswd(otp) + self._console.setTimeout(seconds) + self._console.open() + hooks.before_vm_set_ticket(self._lastXMLDesc, self.conf, params) self._dom.updateDeviceFlags(graphics.toxml(), 0) hooks.after_vm_set_ticket(self._lastXMLDesc, self.conf, params) @@ -2041,6 +2076,8 @@ self._vmStats.stop() if self.guestAgent: self.guestAgent.stop() + if self._console: + self._console.close() if self._dom: try: self._dom.destroyFlags(libvirt.VIR_DOMAIN_DESTROY_GRACEFUL) @@ -2219,6 +2256,17 @@ dev['address'] = address dev['alias'] = alias + def _getUnderlyingConsoleDeviceInfo(self): + """ + Obtain console device info from libvirt + """ + consolexml = xml.dom.minidom.parseString(self._lastXMLDesc) \ + .childNodes[0].getElementsByTagName('devices')[0] \ + .getElementsByTagName('console')[0] + + self.conf["consolePath"] = str(consolexml.getElementsByTagName( + 'source')[0].getAttribute('path')) + def _getUnderlyingVideoDeviceInfo(self): """ Obtain video devices info from libvirt. diff --git a/vdsm/supervdsmServer.py b/vdsm/supervdsmServer.py index 8d9ada4..e3966be 100755 --- a/vdsm/supervdsmServer.py +++ b/vdsm/supervdsmServer.py @@ -43,8 +43,8 @@ from supervdsm import _SuperVdsmManager, PIDFILE, ADDRESS from storage.fileUtils import chown, resolveGid, resolveUid from storage.fileUtils import validateAccess as _validateAccess -from vdsm.constants import METADATA_GROUP, METADATA_USER, EXT_UDEVADM, \ - DISKIMAGE_USER, DISKIMAGE_GROUP, P_LIBVIRT_VMCHANNELS +from vdsm.constants import METADATA_GROUP, METADATA_USER, EXT_CHOWN, \ + EXT_UDEVADM, DISKIMAGE_USER, DISKIMAGE_GROUP, P_LIBVIRT_VMCHANNELS from storage.devicemapper import _removeMapping, _getPathsStatus import storage.misc import configNetwork @@ -268,6 +268,18 @@ def removeFs(self, path): return mkimage.removeFs(path) + @logDecorator + def changePtyOwner(self, ptyNum): + if type(ptyNum) != int: + raise TypeError("ptyNum must be integer") + + cmd = [EXT_CHOWN, "vdsm:kvm", "/dev/pts/" + str(ptyNum)] + rc, out, err = storage.misc.execCmd(cmd, sudo=False) + if rc: + raise OSError(errno.EINVAL, "can not change own of /dev/pts/%d, " \ + "out: %s\nerr: %s" % + (ptyNum, out, err)) + def __pokeParent(parentPid): try: diff --git a/vdsm/utils.py b/vdsm/utils.py index 5e2d4e5..f0ebf79 100644 --- a/vdsm/utils.py +++ b/vdsm/utils.py @@ -36,6 +36,7 @@ import fcntl import functools import stat +from array import array import ethtool @@ -829,3 +830,27 @@ def __unicode__(self): return unicode(self.cmd) + + +class Bitmap(object): + + def __init__(self, size): + self._size = size + self._array = array('B', [0] * ((size / 8) + 1)) + + def __len__(self): + return self._size + + def __getitem__(self, key): + if key >= self._size or key < 0: + raise IndexError + return (self._array[key / 8] >> (key % 8)) & 0x1 + + def __setitem__(self, key, value): + if key >= self._size or key < 0: + raise IndexError + p = key / 8 + if value == 0x1: + self._array[p] = self._array[p] | (0x1 << (key % 8)) + else: + self._array[p] = self._array[p] & ~(0x1 << (key % 8)) \ No newline at end of file diff --git a/vdsm/vm.py b/vdsm/vm.py index 44796c3..f2b8e04 100644 --- a/vdsm/vm.py +++ b/vdsm/vm.py @@ -311,6 +311,7 @@ SOUND_DEVICES: [], VIDEO_DEVICES: [], CONTROLLER_DEVICES: [], GENERAL_DEVICES: [], BALLOON_DEVICES: [], REDIR_DEVICES: []} + self._console = None def _get_lastStatus(self): SHOW_PAUSED_STATES = ('Powering down', 'RebootInProgress', 'Up') @@ -971,6 +972,8 @@ stats['cdrom'] = self.conf['cdrom'] if 'boot' in self.conf: stats['boot'] = self.conf['boot'] + if utils.tobool(self.conf.get('enableConsole', False)): + stats['consolePort'] = self.conf['consolePort'] decStats = {} try: -- To view, visit http://gerrit.ovirt.org/7165 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I69904bf7aafd4f764a256d4075c9bf71d988e7c5 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Xu He Jie <xuhj(a)linux.vnet.ibm.com>

8 19

Change in vdsm[master]: move Global api to clientif
by Xu He Jie 26 Dec '12

26 Dec '12

Xu He Jie has uploaded a new change for review. Change subject: move Global api to clientif ...................................................................... move Global api to clientif Change-Id: I02338ed41b5d200fd0ffd9b96499ae449060f235 Signed-off-by: Xu He Jie <xuhj(a)linux.vnet.ibm.com> --- M vdsm/API.py M vdsm/clientIF.py 2 files changed, 446 insertions(+), 355 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/27/5527/1 -- To view, visit http://gerrit.ovirt.org/5527 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I02338ed41b5d200fd0ffd9b96499ae449060f235 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Xu He Jie <xuhj(a)linux.vnet.ibm.com>

2 3

← Newer
1
...
9
10
11
12
13
14
15
...
39
Older →

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

vdsm-patches August 2012