Change in vdsm[master]: Live Merge: Restore watermark tracking

Monday, 18 July 2016

Hello Adam Litke,

I'd like you to do a code review.  Please visit

    https://gerrit.ovirt.org/60889

to review the following change.

Change subject: Live Merge: Restore watermark tracking
......................................................................

Live Merge: Restore watermark tracking

Since libvirt 1.2.13, the virConnectGetAllDomainStats API will return
write watermark information for all volumes in the chain.  We can use
this information during an active live merge operation to perform
on-demand extension of the merge target (just as we already do for the
active layer).  When libvirt does not provide the necessary information,
use a preemptive extension instead.

Change-Id: I52199bb38a2d3f439d33b9e78743c3e611349672
Bug-Url: https://bugzilla.redhat.com/1168327
Signed-off-by: Adam Litke <alitke(a)redhat.com&gt;
Signed-off-by: Ala Hino <ahino(a)redhat.com&gt;
---
M vdsm/virt/vm.py
1 file changed, 72 insertions(+), 13 deletions(-)


  git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/89/60889/1

diff --git a/vdsm/virt/vm.py b/vdsm/virt/vm.py
index 4dd7596..cc50ba7 100644
--- a/vdsm/virt/vm.py
+++ b/vdsm/virt/vm.py
@@ -917,6 +917,7 @@
 
     def _getExtendCandidates(self):
         ret = []
+        mergeCandidates = self._getLiveMergeExtendCandidates()
 
         for drive in self._chunkedDrives():
             try:
@@ -929,6 +930,76 @@
             ret.append((drive, drive.volumeID, capacity, alloc, physical))
 
         return ret
+
+    def _getLiveMergeExtendCandidates(self):
+        # The common case is that there are no active jobs.
+        if not self.conf['_blockJobs'].values():
+            return {}
+
+        candidates = {}
+        try:
+            vm_sample = sampling.stats_cache.get(self.id)
+            stats = vmstats.produce(self,
+                                    vm_sample.first_value,
+                                    vm_sample.last_value,
+                                    vm_sample.interval)
+            watermarks = stats['watermarks']
+            self.log.debug("AHINO: watermarks %s:", watermarks)
+        except Exception:
+            self.log.exception("Error fetching volumes watermark")
+            return {}
+
+        for job in self.conf['_blockJobs'].values():
+            try:
+                drive = self._findDriveByUUIDs(job['disk'])
+            except LookupError:
+                # After an active layer merge completes the vdsm metadata will
+                # be out of sync for a brief period.  If we cannot find the old
+                # disk then it's safe to skip it.
+                self.log.debug("Couldn't find drive %s. After an active"
+                               " layer merge completes the vdsm metadata"
+                               " will be out of sync for a brief period."
+                               " If we cannot find the old disk then it's"
+                               " safe to skip it",
+                               job['disk'])
+                continue
+
+            if not drive.blockDev:
+                continue
+
+            if job['strategy'] == 'commit':
+                volumeID = job['baseVolume']
+            else:
+                self.log.debug("Unrecognized merge strategy '%s'",
+                               job['strategy'])
+                continue
+
+            if volumeID not in watermarks:
+                self.log.warning("No watermark info available for %s",
+                                 volumeID)
+                continue
+
+            res = self.cif.irs.getVolumeInfo(drive.domainID, drive.poolID,
+                                             drive.imageID, volumeID)
+            if res['status']['code'] != 0:
+                self.log.error("Unable to get the info of volume %s (domain: "
+                               "%s image: %s)", volumeID, drive.domainID,
+                               drive.imageID)
+                continue
+            volInfo = res['info']
+            if volInfo['format'].lower() != 'cow':
+                continue
+
+            self.log.debug("Adding live merge extension candidate: "
+                           "volume=%s allocation=%i", volumeID,
+                           watermarks[volumeID])
+            candidates[drive.imageID] = {
+                'alloc': watermarks[volumeID],
+                'physical': int(volInfo['truesize']),
+                'capacity': int(volInfo['apparentsize']),
+                'volumeID': volumeID}
+
+            return candidates
 
     def _chunkedDrives(self):
         """
@@ -4651,20 +4722,8 @@
                 self.untrackBlockJob(jobUUID)
                 return response.error('mergeErr')
 
-        # blockCommit will cause data to be written into the base volume.
-        # Perform an initial extension to ensure there is enough space to
-        # copy all the required data.  Normally we'd use monitoring to extend
-        # the volume on-demand but internal watermark information is not being
-        # reported by libvirt so we must do the full extension up front.  In
-        # the worst case, the allocated size of 'base' should be increased by
-        # the allocated size of 'top' plus one additional chunk to accomodate
-        # additional writes to 'top' during the live merge operation.
         if drive.chunked and baseInfo['format'] == 'COW':
-            capacity, alloc, physical = self._getExtendInfo(drive)
-            baseSize = int(baseInfo['apparentsize'])
-            topSize = int(topInfo['apparentsize'])
-            maxAlloc = baseSize + topSize
-            self.extendDriveVolume(drive, baseVolUUID, maxAlloc, capacity)
+            self.extendDrivesIfNeeded()
 
         # Trigger the collection of stats before returning so that callers
         # of getVmStats after this returns will see the new job


-- 
To view, visit https://gerrit.ovirt.org/60889
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I52199bb38a2d3f439d33b9e78743c3e611349672
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Ala Hino <ahino(a)redhat.com&gt;
Gerrit-Reviewer: Adam Litke <alitke(a)redhat.com&gt;
    

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011