Federico Simoncelli has uploaded a new change for review.
Change subject: image: placeholder optimization on preallocation ......................................................................
image: placeholder optimization on preallocation
When the destination of a copy or move command is an NFS domain we should try to optimize the placeholders creation avoiding unnecessary prezeroing (only relevant case: RAW PREALLOCATED). In such case in the past we were using a TEMPORARY_VOLUME_SIZE to create a placeholder with a temporary size of 10Mb. This is interfering with live storage migration as the following volumes in the chain are inheriting the temporary size inside the qcow header (virtual disk size). With this patch we are instead forcing the creation of the volume as SPARSE as this won't have any side effect on the qcow header.
Change-Id: I32811bd45320ef02bfc593a71dfdafc0b0550c7d Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=910445 Signed-off-by: Federico Simoncelli fsimonce@redhat.com --- M vdsm/storage/blockSD.py M vdsm/storage/image.py M vdsm/storage/sd.py 3 files changed, 30 insertions(+), 12 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/92/12692/1
diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py index 9809bf4..e040596 100644 --- a/vdsm/storage/blockSD.py +++ b/vdsm/storage/blockSD.py @@ -431,6 +431,14 @@ def requiresMailbox(self): return True
+ @property + def lightweightPreallocation(self): + """ + This property advertises whether creating preallocated volumes + is a lightweight operation or not. + """ + return True + def _registerResourceNamespaces(self): """ Register resources namespaces and create diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py index 92021ff..1b6f89d 100644 --- a/vdsm/storage/image.py +++ b/vdsm/storage/image.py @@ -523,18 +523,20 @@ # find out src volume parameters volParams = srcVol.getVolumeParams(bs=1)
- # To avoid 'prezeroing' preallocated volume on NFS domain, - # we create the target volume with minimal size and after - # that w'll change its metadata back to the original size. - if (volParams['volFormat'] == volume.COW_FORMAT - or volParams['prealloc'] == volume.SPARSE_VOL): - volTmpSize = volParams['size'] + # To avoid prezeroing preallocated volumes on NFS domains + # we create the target as a sparse volume (since it will be + # soon filled with the data coming from the copy) and then + # we change its metadata back to the original value. + if (volParams['prealloc'] == volume.PREALLOCATED_VOL + and not destDom.lightweightPreallocation): + tmpVolPreallocation = volume.SPARSE_VOL else: - volTmpSize = TEMPORARY_VOLUME_SIZE # in sectors (10M) + tmpVolPreallocation = volParams['prealloc']
- destDom.createVolume(imgUUID=imgUUID, size=volTmpSize, + destDom.createVolume(imgUUID=imgUUID, + size=volParams['size'], volFormat=volParams['volFormat'], - preallocate=volParams['prealloc'], + preallocate=tmpVolPreallocation, diskType=volParams['disktype'], volUUID=srcVol.volUUID, desc=volParams['descr'], @@ -548,9 +550,9 @@ dstVol.extend((volParams['apparentsize'] + 511) / 512)
# Change destination volume metadata back to the original - # size. - if volTmpSize != volParams['size']: - dstVol.setSize(volParams['size']) + # type. + if tmpVolPreallocation != volParams['prealloc']: + dstVol.setType(volParams['prealloc'])
dstChain.append(dstVol) except se.StorageException: diff --git a/vdsm/storage/sd.py b/vdsm/storage/sd.py index 9ce836b..7006a88 100644 --- a/vdsm/storage/sd.py +++ b/vdsm/storage/sd.py @@ -316,6 +316,14 @@ return False
@property + def lightweightPreallocation(self): + """ + This property advertises whether creating preallocated volumes + is a lightweight operation or not. + """ + return False + + @property def oop(self): return oop.getProcessPool(self.sdUUID)
-- To view, visit http://gerrit.ovirt.org/12692 To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange Gerrit-Change-Id: I32811bd45320ef02bfc593a71dfdafc0b0550c7d Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli fsimonce@redhat.com
Federico Simoncelli has posted comments on this change.
Change subject: image: placeholder optimization on preallocation ......................................................................
Patch Set 1: (1 inline comment)
.................................................... File vdsm/storage/image.py Line 527: # we create the target as a sparse volume (since it will be Line 528: # soon filled with the data coming from the copy) and then Line 529: # we change its metadata back to the original value. Line 530: if (volParams['prealloc'] == volume.PREALLOCATED_VOL Line 531: and not destDom.lightweightPreallocation): Now that I think of it, it might be better a different semantic: "supportsSparseness" and always use SPARSE_VOL when supported. Line 532: tmpVolPreallocation = volume.SPARSE_VOL Line 533: else: Line 534: tmpVolPreallocation = volParams['prealloc'] Line 535:
-- To view, visit http://gerrit.ovirt.org/12692 To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: comment Gerrit-Change-Id: I32811bd45320ef02bfc593a71dfdafc0b0550c7d Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: Ayal Baron abaron@redhat.com Gerrit-Reviewer: Dan Kenigsberg danken@redhat.com Gerrit-Reviewer: Federico Simoncelli fsimonce@redhat.com
oVirt Jenkins CI Server has posted comments on this change.
Change subject: image: placeholder optimization on preallocation ......................................................................
Patch Set 1:
Build Started http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/1436/
-- To view, visit http://gerrit.ovirt.org/12692 To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: comment Gerrit-Change-Id: I32811bd45320ef02bfc593a71dfdafc0b0550c7d Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: Ayal Baron abaron@redhat.com Gerrit-Reviewer: Dan Kenigsberg danken@redhat.com Gerrit-Reviewer: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: oVirt Jenkins CI Server
oVirt Jenkins CI Server has posted comments on this change.
Change subject: image: placeholder optimization on preallocation ......................................................................
Patch Set 1:
Build Started http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit/1474/ (2/2)
-- To view, visit http://gerrit.ovirt.org/12692 To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: comment Gerrit-Change-Id: I32811bd45320ef02bfc593a71dfdafc0b0550c7d Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: Ayal Baron abaron@redhat.com Gerrit-Reviewer: Dan Kenigsberg danken@redhat.com Gerrit-Reviewer: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: oVirt Jenkins CI Server
oVirt Jenkins CI Server has posted comments on this change.
Change subject: image: placeholder optimization on preallocation ......................................................................
Patch Set 1: Fails
Build Failed
http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/1436/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit/1474/ : FAILURE
-- To view, visit http://gerrit.ovirt.org/12692 To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: comment Gerrit-Change-Id: I32811bd45320ef02bfc593a71dfdafc0b0550c7d Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: Ayal Baron abaron@redhat.com Gerrit-Reviewer: Dan Kenigsberg danken@redhat.com Gerrit-Reviewer: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: oVirt Jenkins CI Server
oVirt Jenkins CI Server has posted comments on this change.
Change subject: image: placeholder optimization on preallocation ......................................................................
Patch Set 2:
Build Started http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/1710/ (2/2)
-- To view, visit http://gerrit.ovirt.org/12692 To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: comment Gerrit-Change-Id: I32811bd45320ef02bfc593a71dfdafc0b0550c7d Gerrit-PatchSet: 2 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: Ayal Baron abaron@redhat.com Gerrit-Reviewer: Dan Kenigsberg danken@redhat.com Gerrit-Reviewer: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: oVirt Jenkins CI Server
oVirt Jenkins CI Server has posted comments on this change.
Change subject: image: placeholder optimization on preallocation ......................................................................
Patch Set 2:
Build Started http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit/1760/ (1/2)
-- To view, visit http://gerrit.ovirt.org/12692 To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: comment Gerrit-Change-Id: I32811bd45320ef02bfc593a71dfdafc0b0550c7d Gerrit-PatchSet: 2 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: Ayal Baron abaron@redhat.com Gerrit-Reviewer: Dan Kenigsberg danken@redhat.com Gerrit-Reviewer: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: oVirt Jenkins CI Server
oVirt Jenkins CI Server has posted comments on this change.
Change subject: image: placeholder optimization on preallocation ......................................................................
Patch Set 2:
Build Successful
http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/1710/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit/1760/ : SUCCESS
-- To view, visit http://gerrit.ovirt.org/12692 To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: comment Gerrit-Change-Id: I32811bd45320ef02bfc593a71dfdafc0b0550c7d Gerrit-PatchSet: 2 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: Ayal Baron abaron@redhat.com Gerrit-Reviewer: Dan Kenigsberg danken@redhat.com Gerrit-Reviewer: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: oVirt Jenkins CI Server
Ayal Baron has posted comments on this change.
Change subject: image: placeholder optimization on preallocation ......................................................................
Patch Set 2: Looks good to me, approved
-- To view, visit http://gerrit.ovirt.org/12692 To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: comment Gerrit-Change-Id: I32811bd45320ef02bfc593a71dfdafc0b0550c7d Gerrit-PatchSet: 2 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: Ayal Baron abaron@redhat.com Gerrit-Reviewer: Dan Kenigsberg danken@redhat.com Gerrit-Reviewer: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: oVirt Jenkins CI Server
oVirt Jenkins CI Server has posted comments on this change.
Change subject: image: placeholder optimization on preallocation ......................................................................
Patch Set 3:
Build Started http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit/1841/ (1/2)
-- To view, visit http://gerrit.ovirt.org/12692 To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: comment Gerrit-Change-Id: I32811bd45320ef02bfc593a71dfdafc0b0550c7d Gerrit-PatchSet: 3 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: Ayal Baron abaron@redhat.com Gerrit-Reviewer: Dan Kenigsberg danken@redhat.com Gerrit-Reviewer: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: oVirt Jenkins CI Server
oVirt Jenkins CI Server has posted comments on this change.
Change subject: image: placeholder optimization on preallocation ......................................................................
Patch Set 3:
Build Started http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/1790/ (2/2)
-- To view, visit http://gerrit.ovirt.org/12692 To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: comment Gerrit-Change-Id: I32811bd45320ef02bfc593a71dfdafc0b0550c7d Gerrit-PatchSet: 3 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: Ayal Baron abaron@redhat.com Gerrit-Reviewer: Dan Kenigsberg danken@redhat.com Gerrit-Reviewer: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: oVirt Jenkins CI Server
Federico Simoncelli has posted comments on this change.
Change subject: image: placeholder optimization on preallocation ......................................................................
Patch Set 3: Verified
Verified: current master exposes the issue explained in bz910445. The patch solves the issues. Block storage domains are unaffected (cloneImageStructure is working as before).
-- To view, visit http://gerrit.ovirt.org/12692 To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: comment Gerrit-Change-Id: I32811bd45320ef02bfc593a71dfdafc0b0550c7d Gerrit-PatchSet: 3 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: Ayal Baron abaron@redhat.com Gerrit-Reviewer: Dan Kenigsberg danken@redhat.com Gerrit-Reviewer: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: oVirt Jenkins CI Server
oVirt Jenkins CI Server has posted comments on this change.
Change subject: image: placeholder optimization on preallocation ......................................................................
Patch Set 3:
Build Successful
http://jenkins.ovirt.org/job/vdsm_pep8_gerrit/1790/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit/1841/ : SUCCESS
-- To view, visit http://gerrit.ovirt.org/12692 To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: comment Gerrit-Change-Id: I32811bd45320ef02bfc593a71dfdafc0b0550c7d Gerrit-PatchSet: 3 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: Ayal Baron abaron@redhat.com Gerrit-Reviewer: Dan Kenigsberg danken@redhat.com Gerrit-Reviewer: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: oVirt Jenkins CI Server
Dan Kenigsberg has submitted this change and it was merged.
Change subject: image: placeholder optimization on preallocation ......................................................................
image: placeholder optimization on preallocation
When the destination of a copy or move command is an NFS domain we should try to optimize the placeholders creation avoiding unnecessary prezeroing (only relevant case: RAW PREALLOCATED). In such case in the past we were using a TEMPORARY_VOLUME_SIZE to create a placeholder with a temporary size of 10Mb. This is interfering with live storage migration as the following volumes in the chain are inheriting the temporary size inside the qcow header (virtual disk size). With this patch we are instead forcing the creation of the volume as SPARSE as this won't have any side effect on the qcow header.
Change-Id: I32811bd45320ef02bfc593a71dfdafc0b0550c7d Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=910445 Signed-off-by: Federico Simoncelli fsimonce@redhat.com --- M vdsm/storage/fileSD.py M vdsm/storage/image.py M vdsm/storage/sd.py 3 files changed, 30 insertions(+), 12 deletions(-)
Approvals: Federico Simoncelli: Verified Dan Kenigsberg: Looks good to me, approved
-- To view, visit http://gerrit.ovirt.org/12692 To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: merged Gerrit-Change-Id: I32811bd45320ef02bfc593a71dfdafc0b0550c7d Gerrit-PatchSet: 3 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: Ayal Baron abaron@redhat.com Gerrit-Reviewer: Dan Kenigsberg danken@redhat.com Gerrit-Reviewer: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: oVirt Jenkins CI Server
Dan Kenigsberg has posted comments on this change.
Change subject: image: placeholder optimization on preallocation ......................................................................
Patch Set 3: Looks good to me, approved
Copying Ayal's +2.
-- To view, visit http://gerrit.ovirt.org/12692 To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: comment Gerrit-Change-Id: I32811bd45320ef02bfc593a71dfdafc0b0550c7d Gerrit-PatchSet: 3 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: Ayal Baron abaron@redhat.com Gerrit-Reviewer: Dan Kenigsberg danken@redhat.com Gerrit-Reviewer: Federico Simoncelli fsimonce@redhat.com Gerrit-Reviewer: oVirt Jenkins CI Server
vdsm-patches@lists.fedorahosted.org