Francesco Romani has uploaded a new change for review.
Change subject: vm: migration: exponential downtime increment ......................................................................
vm: migration: exponential downtime increment
Migration downtime is calculated using an interpolation using the current downtime step (up to a configurabile maximum) and the maximum downtime.
The downtime is incremented at each downtime step until it reaches the maximum, or the migration is finished.
This patch changes the interpolation formula from linear to exponential, being the new one:
d = c ** s
where d is the downtime, s is the step and c is the exponentiation coefficient computed in such a way of when s = S (number of steps to perform), then we will have d = D (selected downtime).
Please note that this patch still preserve an linear increment of the wait time.
Change-Id: I6401772f52ea28144452e67198bddff18f6703eb Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1004101 Signed-off-by: Francesco Romani fromani@redhat.com --- M vdsm/vm.py 1 file changed, 2 insertions(+), 1 deletion(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/20/25820/1
diff --git a/vdsm/vm.py b/vdsm/vm.py index c53f1d4..1d7b643 100644 --- a/vdsm/vm.py +++ b/vdsm/vm.py @@ -722,6 +722,7 @@ delay_per_gib = config.getint('vars', 'migration_downtime_delay') memSize = int(vm.conf['memSize']) self._wait = (delay_per_gib * max(memSize, 2048) + 1023) / 1024 + self._coeff = self._downtime ** (1 / float(self.DOWNTIME_STEPS))
self.daemon = True self.start() @@ -735,7 +736,7 @@ if self._stop.isSet(): break
- downtime = self._downtime * (i + 1) / self.DOWNTIME_STEPS + downtime = int(self._coeff ** i) self._vm.log.debug('setting migration downtime to %d', downtime) self._vm._dom.migrateSetMaxDowntime(downtime, 0)
oVirt Jenkins CI Server has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 1:
Build Successful
http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit_el/6692/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/7483/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit/7593/ : SUCCESS
Michal Skrivanek has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 1:
I wonder if we can factor in the convergence/progress of the migration as well. E.g. for a case when there's a lot to transfer but no trashing we actually would be able to finish migration with low downtime, but because of the time it takes to transfer the initial amount of data we go through several downtime_steps.
Something like having a threshold like "when between two progress updates the change is >10%, don't change the downtime"
Francesco Romani has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 1:
I like the concept, I'll give it a try. At first glance it should be feasible with a few more lines.
Francesco Romani has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 2:
Rebased
oVirt Jenkins CI Server has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 2:
Build Successful
http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/7931/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit_el/7141/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_unit_tests_gerrit/8043/ : SUCCESS
Itamar Heim has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 2:
ping
Francesco Romani has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 3:
rebased
oVirt Jenkins CI Server has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 3:
Build Successful
http://jenkins.ovirt.org/job/vdsm_master_unit_tests_gerrit_el/8889/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/9673/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_virt_functional_tests_gerrit/669/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_unit_tests_gerrit/9828/ : SUCCESS
Itamar Heim has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 3:
ping
Michal Skrivanek has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 3: Code-Review+1
I think this would help in all cases and so we should change the default behavior this way
Francesco Romani has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 3: Verified+1
Verified doing migration among two hosts: one with vanilla VDSM, one with patched VDSM. Inspected the downtime progression though logs:
Old progression: Thread-71::DEBUG::2014-08-29 15:59:56,982::migration::360::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::migration downtime thread started Thread-70::DEBUG::2014-08-29 16:00:20,905::migration::375::vm.Vm::(cancel) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::canceling migration downtime thread Thread-71::DEBUG::2014-08-29 16:00:20,906::migration::372::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::migration downtime thread exiting Thread-97::DEBUG::2014-08-29 16:24:07,093::migration::360::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::migration downtime thread started Thread-97::DEBUG::2014-08-29 16:24:37,094::migration::369::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 50 Thread-97::DEBUG::2014-08-29 16:25:07,097::migration::369::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 100 Thread-97::DEBUG::2014-08-29 16:25:37,100::migration::369::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 150 Thread-97::DEBUG::2014-08-29 16:26:07,103::migration::369::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 200 Thread-97::DEBUG::2014-08-29 16:26:37,106::migration::369::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 250 Thread-97::DEBUG::2014-08-29 16:27:07,109::migration::369::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 300 Thread-97::DEBUG::2014-08-29 16:27:37,112::migration::369::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 350 Thread-97::DEBUG::2014-08-29 16:28:07,116::migration::369::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 400 Thread-96::DEBUG::2014-08-29 16:28:27,684::migration::375::vm.Vm::(cancel) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::canceling migration downtime thread Thread-97::DEBUG::2014-08-29 16:28:27,684::migration::372::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::migration downtime thread exiting
New progression: Thread-36::DEBUG::2014-08-29 16:19:10,845::migration::375::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::migration downtime thread started Thread-36::DEBUG::2014-08-29 16:19:40,846::migration::384::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 1 Thread-36::DEBUG::2014-08-29 16:20:10,858::migration::384::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 1 Thread-36::DEBUG::2014-08-29 16:20:40,859::migration::384::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 3 Thread-36::DEBUG::2014-08-29 16:21:10,861::migration::384::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 6 Thread-36::DEBUG::2014-08-29 16:21:40,863::migration::384::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 12 Thread-36::DEBUG::2014-08-29 16:22:10,865::migration::384::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 22 Thread-36::DEBUG::2014-08-29 16:22:40,891::migration::384::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 41 Thread-36::DEBUG::2014-08-29 16:23:10,908::migration::384::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 77 Thread-35::DEBUG::2014-08-29 16:23:33,317::migration::390::vm.Vm::(cancel) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::canceling migration downtime thread Thread-36::DEBUG::2014-08-29 16:23:33,318::migration::387::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::migration downtime thread exiting
Francesco Romani has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 3:
another simpler migration:
old algorhythm Thread-101::DEBUG::2014-08-29 16:29:11,355::migration::360::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::migration downtime thread started Thread-101::DEBUG::2014-08-29 16:29:41,356::migration::369::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 50 Thread-101::DEBUG::2014-08-29 16:30:11,358::migration::369::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 100 Thread-100::DEBUG::2014-08-29 16:30:29,243::migration::375::vm.Vm::(cancel) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::canceling migration downtime thread Thread-101::DEBUG::2014-08-29 16:30:29,244::migration::372::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::migration downtime thread exiting
new one: Thread-56::DEBUG::2014-08-29 16:30:40,284::migration::375::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::migration downtime thread started Thread-56::DEBUG::2014-08-29 16:31:10,287::migration::384::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 1 Thread-56::DEBUG::2014-08-29 16:31:40,306::migration::384::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 1 Thread-55::DEBUG::2014-08-29 16:31:54,824::migration::390::vm.Vm::(cancel) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::canceling migration downtime thread Thread-56::DEBUG::2014-08-29 16:31:54,827::migration::387::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::migration downtime thread exiting
Francesco Romani has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 3:
Formatting got completely lost. Let's retry.
First migration, longish.
Old sequence: 50,100,150,200,250,300,350,400 New sequence: 1, 1, 3, 6, 12, 22, 41, 77
Second migration, short and simple:
Old sequence: 50,100 New sequence: 1, 1
Dan Kenigsberg has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 3: Code-Review-1
(3 comments)
http://gerrit.ovirt.org/#/c/25820/3//COMMIT_MSG Commit Message:
Line 18: Line 19: d = c ** s Line 20: Line 21: where d is the downtime, s is the step and c is the Line 22: exponentiation coefficient computed in such a way sorry for my puritanism, but in mathematics, "c" is called the *base* of the exponentiation. "Coefficient" implies multiplication. Line 23: of when s = S (number of steps to perform), Line 24: then we will have d = D (selected downtime). Line 25: Line 26: Please note that this patch still preserve an linear
Line 23: of when s = S (number of steps to perform), Line 24: then we will have d = D (selected downtime). Line 25: Line 26: Please note that this patch still preserve an linear Line 27: increment of the wait time. Could you explain why a exponential growth is better than linear? Line 28: Line 29: Change-Id: I6401772f52ea28144452e67198bddff18f6703eb Line 30: Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1004101
Line 26: Please note that this patch still preserve an linear Line 27: increment of the wait time. Line 28: Line 29: Change-Id: I6401772f52ea28144452e67198bddff18f6703eb Line 30: Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1004101 Why is that bug secret? Please make it public, or better explain the need for this patch.
Francesco Romani has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 3:
(3 comments)
http://gerrit.ovirt.org/#/c/25820/3//COMMIT_MSG Commit Message:
Line 18: Line 19: d = c ** s Line 20: Line 21: where d is the downtime, s is the step and c is the Line 22: exponentiation coefficient computed in such a way
sorry for my puritanism, but in mathematics, "c" is called the *base* of th
Your puritanism is very welcome. Will fix. Line 23: of when s = S (number of steps to perform), Line 24: then we will have d = D (selected downtime). Line 25: Line 26: Please note that this patch still preserve an linear
Line 23: of when s = S (number of steps to perform), Line 24: then we will have d = D (selected downtime). Line 25: Line 26: Please note that this patch still preserve an linear Line 27: increment of the wait time.
Could you explain why a exponential growth is better than linear?
Sure, will amend the commit message. Line 28: Line 29: Change-Id: I6401772f52ea28144452e67198bddff18f6703eb Line 30: Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1004101
Line 26: Please note that this patch still preserve an linear Line 27: increment of the wait time. Line 28: Line 29: Change-Id: I6401772f52ea28144452e67198bddff18f6703eb Line 30: Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1004101
Why is that bug secret? Please make it public, or better explain the need f
I'll start with better explanation.
Francesco Romani has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 4: Verified+1
Improved commit message and rebased. No code changes, copied score.
oVirt Jenkins CI Server has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 4:
Build Failed
http://jenkins.ovirt.org/job/vdsm_master_unit_tests_gerrit_el/11224/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/12166/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/12013/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_virt_functional_tests_gerrit/1592/ : There was an infra issue, please contact infra@ovirt.org
Dan Kenigsberg has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 4: Code-Review-1
(3 comments)
http://gerrit.ovirt.org/#/c/25820/4//COMMIT_MSG Commit Message:
Line 26: The rationale for this patch is to minimize the Line 27: downtime of VMs pushing for low downtime values as much Line 28: and as long as possible. Line 29: This way, the downtime becomes more tightly related Line 30: to the duration of the migration itself. please explain better why waiting for 11 seconds after waiting for 10 seconds is not very smart. Line 31: Line 32: Please note that this patch still preserve an linear Line 33: increment of the wait time. Line 34:
http://gerrit.ovirt.org/#/c/25820/4/vdsm/virt/migration.py File vdsm/virt/migration.py:
Line 365: Line 366: delay_per_gib = config.getint('vars', 'migration_downtime_delay') Line 367: memSize = int(vm.conf['memSize']) Line 368: self._wait = (delay_per_gib * max(memSize, 2048) + 1023) / 1024 Line 369: self._coeff = self._downtime ** (1 / float(self.DOWNTIME_STEPS)) _coeff -> _base Line 370: Line 371: self.daemon = True Line 372: self.start() Line 373:
Line 379: Line 380: if self._stop.isSet(): Line 381: break Line 382: Line 383: downtime = int(self._coeff ** i) make sure we don't begin with 0. Line 384: self._vm.log.debug('setting migration downtime to %d', downtime) Line 385: self._vm._dom.migrateSetMaxDowntime(downtime, 0) Line 386: Line 387: self._vm.log.debug('migration downtime thread exiting')
Francesco Romani has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 4: -Verified
(3 comments)
http://gerrit.ovirt.org/#/c/25820/4//COMMIT_MSG Commit Message:
Line 26: The rationale for this patch is to minimize the Line 27: downtime of VMs pushing for low downtime values as much Line 28: and as long as possible. Line 29: This way, the downtime becomes more tightly related Line 30: to the duration of the migration itself.
please explain better why waiting for 11 seconds after waiting for 10 secon
I'll quote David words, since his explanation is alrady pretty good. Line 31: Line 32: Please note that this patch still preserve an linear Line 33: increment of the wait time. Line 34:
http://gerrit.ovirt.org/#/c/25820/4/vdsm/virt/migration.py File vdsm/virt/migration.py:
Line 365: Line 366: delay_per_gib = config.getint('vars', 'migration_downtime_delay') Line 367: memSize = int(vm.conf['memSize']) Line 368: self._wait = (delay_per_gib * max(memSize, 2048) + 1023) / 1024 Line 369: self._coeff = self._downtime ** (1 / float(self.DOWNTIME_STEPS))
_coeff -> _base
Done Line 370: Line 371: self.daemon = True Line 372: self.start() Line 373:
Line 379: Line 380: if self._stop.isSet(): Line 381: break Line 382: Line 383: downtime = int(self._coeff ** i)
make sure we don't begin with 0.
Fixed Line 384: self._vm.log.debug('setting migration downtime to %d', downtime) Line 385: self._vm._dom.migrateSetMaxDowntime(downtime, 0) Line 386: Line 387: self._vm.log.debug('migration downtime thread exiting')
oVirt Jenkins CI Server has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 5:
Build Failed
http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/12360/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_unit_tests_gerrit_el/11416/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_virt_functional_tests_gerrit/1643/ : There was an infra issue, please contact infra@ovirt.org
http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/12206/ : SUCCESS
oVirt Jenkins CI Server has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 6:
Build Failed
http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/12365/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_unit_tests_gerrit_el/11421/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_virt_functional_tests_gerrit/1645/ : There was an infra issue, please contact infra@ovirt.org
http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/12210/ : SUCCESS
Dan Kenigsberg has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 6: Code-Review-1
(1 comment)
http://gerrit.ovirt.org/#/c/25820/6/vdsm/virt/migration.py File vdsm/virt/migration.py:
Line 353: if not self._preparingMigrationEvt: Line 354: raise Line 355: Line 356: Line 357: def exponential_downtime(downtime, steps): would you add a unit test for this function?
list(exponential_downtime(500, 10))
[451, 451, 452, 453, 454, 457, 460, 465, 472, 483]
makes very little sense to me. Line 358: offset = downtime - (downtime / steps) Line 359: base = (downtime - offset) ** (1 / float(steps)) Line 360: Line 361: for i in range(steps):
Francesco Romani has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 6:
(1 comment)
http://gerrit.ovirt.org/#/c/25820/6/vdsm/virt/migration.py File vdsm/virt/migration.py:
Line 353: if not self._preparingMigrationEvt: Line 354: raise Line 355: Line 356: Line 357: def exponential_downtime(downtime, steps):
would you add a unit test for this function?
Because you're right, and I had a bug in my formula. Better to quickly hide this from my calculus professor :) Line 358: offset = downtime - (downtime / steps) Line 359: base = (downtime - offset) ** (1 / float(steps)) Line 360: Line 361: for i in range(steps):
oVirt Jenkins CI Server has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 7:
Build Failed
http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/12372/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_unit_tests_gerrit_el/11428/ : FAILURE
http://jenkins.ovirt.org/job/vdsm_master_virt_functional_tests_gerrit/1647/ : There was an infra issue, please contact infra@ovirt.org
http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/12217/ : SUCCESS
Francesco Romani has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 7: Verified+1
Re-verified running a migration in the following scenario:
- patched VDSM - set vars.migration_max_time_per_gib_mem = 0 - runned 'stress -v -m X' inside the VM to have memory stress - performed migration
expected output:
GENji> 11:38:59 root [~]$ grep -i Downtime /var/log/vdsm//vdsm.log Thread-39::DEBUG::2014-09-10 11:34:15,209::migration::383::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::migration downtime thread started Thread-39::DEBUG::2014-09-10 11:34:45,210::migration::392::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 51 Thread-39::DEBUG::2014-09-10 11:35:15,213::migration::392::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 53 Thread-39::DEBUG::2014-09-10 11:35:45,216::migration::392::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 56 Thread-39::DEBUG::2014-09-10 11:36:15,219::migration::392::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 61 Thread-39::DEBUG::2014-09-10 11:36:45,221::migration::392::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 71 Thread-39::DEBUG::2014-09-10 11:37:15,224::migration::392::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 89 Thread-39::DEBUG::2014-09-10 11:37:45,227::migration::392::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 121 Thread-39::DEBUG::2014-09-10 11:38:15,230::migration::392::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 182 Thread-39::DEBUG::2014-09-10 11:38:45,232::migration::392::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 294 Thread-39::DEBUG::2014-09-10 11:39:15,235::migration::392::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::setting migration downtime to 499 Thread-39::DEBUG::2014-09-10 11:39:15,238::migration::395::vm.Vm::(run) vmId=`56d1c657-dd76-4609-a207-c050699be5be`::migration downtime thread exiting
- migration eventually was detected as stuck and aborted, but that's OK here.
Martin Polednik has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 7: Code-Review+1
(1 comment)
Seems correct to me, +1 for patches (to think: run tests against multiple sets of (downtime, steps) in order to be really sure the properties hold still or support a proof ;))
http://gerrit.ovirt.org/#/c/25820/7/vdsm/virt/migration.py File vdsm/virt/migration.py:
Line 358: offset = downtime / steps Line 359: base = (downtime - offset) ** (1 / float(steps)) Line 360: Line 361: for i in range(1, steps+1): Line 362: yield offset + int(base ** i) Minor: I'd go with yield int(offset + base ** i) in order to guarantee the return value being int even if downtime ∈ ℝ(really minor imho, I'd guess that might destroy something else) Line 363: Line 364: Line 365: class DowntimeThread(threading.Thread): Line 366: DOWNTIME_STEPS = config.getint('vars', 'migration_downtime_steps')
Martin Polednik has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 7:
(1 comment)
http://gerrit.ovirt.org/#/c/25820/7/tests/vmMigrationTests.py File tests/vmMigrationTests.py:
Line 33: return izip(a, b) Line 34: Line 35: Line 36: def _linear_downtime(downtime, steps): Line 37: for i in range(steps): maybe range(1, steps + 1) and later downtime * i / steps for consistence with exponential downtime (also allows easier plotting, for example "plot (d / s) + ((d - d / s) ^ (1 / s)) ^ k and d * k / s for (d = 500, s = 10), (k from 1 to 10)" in wolfram alpha) Line 38: yield downtime * (i + 1) / steps Line 39: Line 40: Line 41: class TestVmMigrationDowntimeSequence(TestCaseBase):
Dan Kenigsberg has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 7: Code-Review-1
(6 comments)
http://gerrit.ovirt.org/#/c/25820/7/tests/vmMigrationTests.py File tests/vmMigrationTests.py:
Line 38: yield downtime * (i + 1) / steps Line 39: Line 40: Line 41: class TestVmMigrationDowntimeSequence(TestCaseBase): Line 42: # defaults as per config read them from config, instead of having this comment and duplicating the value. Line 43: _DOWNTIME = 500 Line 44: Line 45: _STEPS = 10 Line 46:
Line 61: self.assertTrue(len(self._default) >= 2) Line 62: Line 63: def test_downtime_increasing(self): Line 64: for a, b in pairwise(self._default): Line 65: self.assertTrue(a <= b) assertLessEqual() etc gives more info upon failure - please use more specific asserts when possible. Line 66: Line 67: def test_exponential_dowtime_never_zero(self): Line 68: for dt in self._default: Line 69: self.assertTrue(dt > 0)
Line 78: self.assertTrue(a <= b) Line 79: Line 80: def test_exponential_same_end_value(self): Line 81: self.assertAlmostEqual(self._default[-1], self._linear[-1], Line 82: delta=self._DELTA) You'd also want to assert that the final step is the maximum allow downtime.
http://gerrit.ovirt.org/#/c/25820/7/vdsm/virt/migration.py File vdsm/virt/migration.py:
Line 354: raise Line 355: Line 356: Line 357: def exponential_downtime(downtime, steps): Line 358: offset = downtime / steps Let's do a float division on Python2, too. Line 359: base = (downtime - offset) ** (1 / float(steps)) Line 360: Line 361: for i in range(1, steps+1): Line 362: yield offset + int(base ** i)
Line 357: def exponential_downtime(downtime, steps): Line 358: offset = downtime / steps Line 359: base = (downtime - offset) ** (1 / float(steps)) Line 360: Line 361: for i in range(1, steps+1): use spaces around the + sign. Line 362: yield offset + int(base ** i) Line 363: Line 364: Line 365: class DowntimeThread(threading.Thread):
Line 358: offset = downtime / steps Line 359: base = (downtime - offset) ** (1 / float(steps)) Line 360: Line 361: for i in range(1, steps+1): Line 362: yield offset + int(base ** i)
Minor: I'd go with yield int(offset + base ** i) in order to guarantee the
+1, particularly since on Python 3, (downtime / steps) may be in ℝ∖ℕ, anyway. Line 363: Line 364: Line 365: class DowntimeThread(threading.Thread): Line 366: DOWNTIME_STEPS = config.getint('vars', 'migration_downtime_steps')
Francesco Romani has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 7: -Verified
(7 comments)
-Verified because needs more work.
http://gerrit.ovirt.org/#/c/25820/7/tests/vmMigrationTests.py File tests/vmMigrationTests.py:
Line 33: return izip(a, b) Line 34: Line 35: Line 36: def _linear_downtime(downtime, steps): Line 37: for i in range(steps):
maybe range(1, steps + 1) and later downtime * i / steps for consistence wi
Good point, will do. Line 38: yield downtime * (i + 1) / steps Line 39: Line 40: Line 41: class TestVmMigrationDowntimeSequence(TestCaseBase):
Line 38: yield downtime * (i + 1) / steps Line 39: Line 40: Line 41: class TestVmMigrationDowntimeSequence(TestCaseBase): Line 42: # defaults as per config
read them from config, instead of having this comment and duplicating the v
Sure, will fix. Line 43: _DOWNTIME = 500 Line 44: Line 45: _STEPS = 10 Line 46:
Line 61: self.assertTrue(len(self._default) >= 2) Line 62: Line 63: def test_downtime_increasing(self): Line 64: for a, b in pairwise(self._default): Line 65: self.assertTrue(a <= b)
assertLessEqual() etc gives more info upon failure - please use more specif
I'd love to, but
https://docs.python.org/2/library/unittest.html#unittest.TestCase.assertLess... -> new in version 2.7
And AFAIK it wasn't backported (git grep confirms) Line 66: Line 67: def test_exponential_dowtime_never_zero(self): Line 68: for dt in self._default: Line 69: self.assertTrue(dt > 0)
Line 78: self.assertTrue(a <= b) Line 79: Line 80: def test_exponential_same_end_value(self): Line 81: self.assertAlmostEqual(self._default[-1], self._linear[-1], Line 82: delta=self._DELTA)
You'd also want to assert that the final step is the maximum allow downtime
right, I'll add.
http://gerrit.ovirt.org/#/c/25820/7/vdsm/virt/migration.py File vdsm/virt/migration.py:
Line 354: raise Line 355: Line 356: Line 357: def exponential_downtime(downtime, steps): Line 358: offset = downtime / steps
Let's do a float division on Python2, too.
will fix Line 359: base = (downtime - offset) ** (1 / float(steps)) Line 360: Line 361: for i in range(1, steps+1): Line 362: yield offset + int(base ** i)
Line 357: def exponential_downtime(downtime, steps): Line 358: offset = downtime / steps Line 359: base = (downtime - offset) ** (1 / float(steps)) Line 360: Line 361: for i in range(1, steps+1):
use spaces around the + sign.
oops, will fix. Line 362: yield offset + int(base ** i) Line 363: Line 364: Line 365: class DowntimeThread(threading.Thread):
Line 358: offset = downtime / steps Line 359: base = (downtime - offset) ** (1 / float(steps)) Line 360: Line 361: for i in range(1, steps+1): Line 362: yield offset + int(base ** i)
+1, particularly since on Python 3, (downtime / steps) may be in ℝ∖ℕ, anywa
Good point, that's exactly the comment I was looking for :) Thanks! Line 363: Line 364: Line 365: class DowntimeThread(threading.Thread): Line 366: DOWNTIME_STEPS = config.getint('vars', 'migration_downtime_steps')
Francesco Romani has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 7:
(1 comment)
http://gerrit.ovirt.org/#/c/25820/7/tests/vmMigrationTests.py File tests/vmMigrationTests.py:
Line 33: return izip(a, b) Line 34: Line 35: Line 36: def _linear_downtime(downtime, steps): Line 37: for i in range(steps):
Good point, will do.
Sorry, I overlooked. This code needs to as the old one as reference, so I'm keeping it for this reason. Line 38: yield downtime * (i + 1) / steps Line 39: Line 40: Line 41: class TestVmMigrationDowntimeSequence(TestCaseBase):
Martin Polednik has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 7:
(1 comment)
http://gerrit.ovirt.org/#/c/25820/7/tests/vmMigrationTests.py File tests/vmMigrationTests.py:
Line 33: return izip(a, b) Line 34: Line 35: Line 36: def _linear_downtime(downtime, steps): Line 37: for i in range(steps):
Sorry, I overlooked. This code needs to as the old one as reference, so I'm
what about leaving a comment that these are equivalent? It's not hard to understand *now*, but later it might bring questions Line 38: yield downtime * (i + 1) / steps Line 39: Line 40: Line 41: class TestVmMigrationDowntimeSequence(TestCaseBase):
Francesco Romani has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 7:
(1 comment)
http://gerrit.ovirt.org/#/c/25820/7/tests/vmMigrationTests.py File tests/vmMigrationTests.py:
Line 33: return izip(a, b) Line 34: Line 35: Line 36: def _linear_downtime(downtime, steps): Line 37: for i in range(steps):
what about leaving a comment that these are equivalent? It's not hard to un
Done! Line 38: yield downtime * (i + 1) / steps Line 39: Line 40: Line 41: class TestVmMigrationDowntimeSequence(TestCaseBase):
Francesco Romani has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 8:
improved the tests, most notably adding permutations smaller tuning to the exponential function after the testing
oVirt Jenkins CI Server has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 8:
Build Failed
http://jenkins.ovirt.org/job/vdsm_master_unit_tests_gerrit_el/11535/ : FAILURE
http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/12479/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_virt_functional_tests_gerrit/1671/ : There was an infra issue, please contact infra@ovirt.org
http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/12324/ : SUCCESS
Martin Polednik has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 8: Code-Review-1
(5 comments)
-1 for visibility, other than the minor things it seems to be OK (+ need answer for the steps >= 2 question)
http://gerrit.ovirt.org/#/c/25820/8/tests/vmMigrationTests.py File tests/vmMigrationTests.py:
Line 46: class TestVmMigrationDowntimeSequence(TestCaseBase): Line 47: Line 48: @permutations(_PARAMS) Line 49: def test_downtime_is_sequence(self, dtime, steps): Line 50: self.assertTrue(len(self._default(dtime, steps)) >= 2) can be rewritten as self.assertGreaterEqual(len(self._default(dtime, steps)), 2)
another thing, is steps really >= 2 or do we allow single step? Line 51: Line 52: @permutations(_PARAMS) Line 53: def test_downtime_increasing(self, dtime, steps): Line 54: for a, b in pairwise(self._default(dtime, steps)):
Line 51: Line 52: @permutations(_PARAMS) Line 53: def test_downtime_increasing(self, dtime, steps): Line 54: for a, b in pairwise(self._default(dtime, steps)): Line 55: self.assertTrue(a <= b) can be rewritten as self.assertGreater(b, a) Line 56: Line 57: @permutations(_PARAMS) Line 58: def test_exponential_dowtime_never_zero(self, dtime, steps): Line 59: for dt in self._default(dtime, steps):
Line 56: Line 57: @permutations(_PARAMS) Line 58: def test_exponential_dowtime_never_zero(self, dtime, steps): Line 59: for dt in self._default(dtime, steps): Line 60: self.assertTrue(dt > 0) can be rewritten as self.assertGreater(dt, 0) Line 61: Line 62: @permutations(_PARAMS) Line 63: def test_exponential_downtime_is_lower(self, dtime, steps): Line 64: # it's OK if exponential starts a little higher than linear...
Line 69: Line 70: # ...but what matters is that after that, it stays lower. Line 71: for i, (a, b) in enumerate(zip(exp[1:], lin[1:])): Line 72: msg = 'step=%i/%i exp=%f lin=%f' % (i+1, steps, a, b) Line 73: self.assertTrue(a <= b, msg) again, self assertGreater(b, a, msg) Line 74: Line 75: @permutations(_PARAMS) Line 76: def test_exponential_same_end_value(self, dtime, steps): Line 77: exp = self._default(dtime, steps)
Line 107: def pairwise(iterable): Line 108: "s -> (s0,s1), (s1,s2), (s2, s3), ..." Line 109: a, b = tee(iterable) Line 110: next(b, None) Line 111: return izip(a, b) thought about this a bit, wouldn't importing islice and running izip(a, islice(b, 1, None)) be a bit nicer? Line 112: Line 113: Line 114: def _linear_downtime(downtime, steps): Line 115: "this is the old formula as reference"
Dan Kenigsberg has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 8: Code-Review-1
(3 comments)
minor nit
http://gerrit.ovirt.org/#/c/25820/8/tests/vmMigrationTests.py File tests/vmMigrationTests.py:
Line 46: class TestVmMigrationDowntimeSequence(TestCaseBase): Line 47: Line 48: @permutations(_PARAMS) Line 49: def test_downtime_is_sequence(self, dtime, steps): Line 50: self.assertTrue(len(self._default(dtime, steps)) >= 2)
can be rewritten as self.assertGreaterEqual(len(self._default(dtime, steps)
As Francesco noted to me, these were introduced only on Python 2.7 (we still need to support 2.6). Line 51: Line 52: @permutations(_PARAMS) Line 53: def test_downtime_increasing(self, dtime, steps): Line 54: for a, b in pairwise(self._default(dtime, steps)):
Line 107: def pairwise(iterable): Line 108: "s -> (s0,s1), (s1,s2), (s2, s3), ..." Line 109: a, b = tee(iterable) Line 110: next(b, None) Line 111: return izip(a, b)
thought about this a bit, wouldn't importing islice and running izip(a, isl
I don't really understand your suggestion; I like copying the recipe unless there's a good reason not to. Line 112: Line 113: Line 114: def _linear_downtime(downtime, steps): Line 115: "this is the old formula as reference"
http://gerrit.ovirt.org/#/c/25820/8/vdsm/virt/migration.py File vdsm/virt/migration.py:
Line 355: Line 356: Line 357: def exponential_downtime(downtime, steps): Line 358: offset = downtime / float(steps) Line 359: base = (downtime - offset) ** (1 / float(steps-1)) space around operator Line 360: Line 361: for i in range(steps): Line 362: yield int(offset + base ** i) Line 363:
oVirt Jenkins CI Server has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 9:
Build Failed
http://jenkins.ovirt.org/job/vdsm_master_unit_tests_gerrit_el/11759/ : FAILURE
http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/12703/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_virt_functional_tests_gerrit/1707/ : There was an infra issue, please contact infra@ovirt.org
http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/12548/ : SUCCESS
Dan Kenigsberg has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 9: Code-Review+2
Martin Polednik has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 9: Code-Review+1
Francesco Romani has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 9: Verified+1
re-verified while running a few migration, both with idle and busy VM:
Thread-444::DEBUG::2014-11-04 11:49:30,905::migration::393::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::migration downtime thread started Thread-444::DEBUG::2014-11-04 11:50:00,916::migration::405::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::setting migration downtime to 51 Thread-444::DEBUG::2014-11-04 11:50:30,928::migration::405::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::setting migration downtime to 51 Thread-444::DEBUG::2014-11-04 11:51:00,933::migration::405::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::setting migration downtime to 53 Thread-444::DEBUG::2014-11-04 11:51:30,937::migration::405::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::setting migration downtime to 57 Thread-444::DEBUG::2014-11-04 11:52:00,942::migration::405::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::setting migration downtime to 65 Thread-444::DEBUG::2014-11-04 11:52:30,949::migration::405::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::setting migration downtime to 79 Thread-444::DEBUG::2014-11-04 11:53:00,953::migration::405::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::setting migration downtime to 108 Thread-444::DEBUG::2014-11-04 11:53:30,957::migration::405::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::setting migration downtime to 165 Thread-443::DEBUG::2014-11-04 11:53:51,288::migration::411::vm.Vm::(stop) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::stopping migration downtime thread Thread-444::DEBUG::2014-11-04 11:53:51,289::migration::408::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::migration downtime thread exiting Thread-733::DEBUG::2014-11-04 11:56:48,123::migration::393::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::migration downtime thread started Thread-733::DEBUG::2014-11-04 11:57:18,127::migration::405::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::setting migration downtime to 51 Thread-733::DEBUG::2014-11-04 11:57:48,131::migration::405::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::setting migration downtime to 51 Thread-733::DEBUG::2014-11-04 11:58:18,136::migration::405::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::setting migration downtime to 53 Thread-733::DEBUG::2014-11-04 11:58:48,140::migration::405::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::setting migration downtime to 57 Thread-732::DEBUG::2014-11-04 11:59:00,561::migration::411::vm.Vm::(stop) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::stopping migration downtime thread Thread-733::DEBUG::2014-11-04 11:59:00,564::migration::408::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::migration downtime thread exiting Thread-864::DEBUG::2014-11-04 11:59:45,684::migration::393::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::migration downtime thread started Thread-864::DEBUG::2014-11-04 12:00:15,687::migration::405::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::setting migration downtime to 51 Thread-864::DEBUG::2014-11-04 12:00:45,691::migration::405::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::setting migration downtime to 51 Thread-864::DEBUG::2014-11-04 12:01:15,696::migration::405::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::setting migration downtime to 53 Thread-864::DEBUG::2014-11-04 12:01:45,700::migration::405::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::setting migration downtime to 57 Thread-863::DEBUG::2014-11-04 12:01:56,836::migration::411::vm.Vm::(stop) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::stopping migration downtime thread Thread-864::DEBUG::2014-11-04 12:01:56,839::migration::408::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::migration downtime thread exiting Thread-1065::DEBUG::2014-11-04 12:04:37,643::migration::393::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::migration downtime thread started Thread-1064::DEBUG::2014-11-04 12:04:56,465::migration::411::vm.Vm::(stop) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::stopping migration downtime thread Thread-1065::DEBUG::2014-11-04 12:04:56,468::migration::408::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::migration downtime thread exiting Thread-1108::DEBUG::2014-11-04 12:05:36,462::migration::393::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::migration downtime thread started Thread-1105::DEBUG::2014-11-04 12:05:57,641::migration::411::vm.Vm::(stop) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::stopping migration downtime thread Thread-1108::DEBUG::2014-11-04 12:05:57,644::migration::408::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::migration downtime thread exiting Thread-1217::DEBUG::2014-11-04 12:08:19,482::migration::393::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::migration downtime thread started Thread-1216::DEBUG::2014-11-04 12:08:39,996::migration::411::vm.Vm::(stop) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::stopping migration downtime thread Thread-1217::DEBUG::2014-11-04 12:08:40,000::migration::408::vm.Vm::(run) vmId=`ace61b23-f455-4fc7-8d59-d9c469657b2a`::migration downtime thread exiting
Dan Kenigsberg has submitted this change and it was merged.
Change subject: vm: migration: exponential downtime increment ......................................................................
vm: migration: exponential downtime increment
Migration downtime is calculated using an interpolation using the current downtime step (up to a configurabile maximum) and the maximum downtime.
The downtime is incremented at each downtime step until it reaches the maximum, or the migration is finished.
This patch changes the interpolation formula from linear to exponential, being the new one:
d = c ** s + k
where d is the downtime, s is the step and c is the exponentiation base computed in such a way of when s = S (number of steps to perform), then we will have d = D (selected downtime). K is a constant added to make sure that downtime 'd' is never equal to zero.
The rationale for this patch is that moving linearly from the minimum to maximum downtime limit isn't very useful: if if we took 1s downtime for a migration that could complete with 0.1s downtime, that's very bad, but if we take 11s to complete a migration that could finish in 10s, it's not a big deal.
So, it makes more sense to step towards the final hard limit exponentially rather than linearly. In addition, because most of the attempts will be much shorted than the final hard limit, it won't blow out the total time to complete migration nearly as badly.
Please note that this patch still preserve an linear increment of the wait time.
Along the way, we take the chance to bootstrap unit tests for vdsm/virt/migration.py.
Change-Id: I6401772f52ea28144452e67198bddff18f6703eb Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1004101 Signed-off-by: Francesco Romani fromani@redhat.com Reviewed-on: http://gerrit.ovirt.org/25820 Reviewed-by: Dan Kenigsberg danken@redhat.com Reviewed-by: Martin Polednik mpolednik@redhat.com --- M tests/Makefile.am A tests/vmMigrationTests.py M vdsm/virt/migration.py 3 files changed, 132 insertions(+), 3 deletions(-)
Approvals: Dan Kenigsberg: Looks good to me, approved Francesco Romani: Verified Martin Polednik: Looks good to me, but someone else must approve
oVirt Jenkins CI Server has posted comments on this change.
Change subject: vm: migration: exponential downtime increment ......................................................................
Patch Set 10:
Build Failed
http://jenkins.ovirt.org/job/vdsm_master_create-rpms-fc21-x86_64_merged/155/ : FAILURE
http://jenkins.ovirt.org/job/vdsm_master_create-rpms-el6-x86_64_merged/178/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_create-rpms_merged_test_debug/371/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_verify-error-codes_merged/6003/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_create-rpms-el7-x86_64_merged/176/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_create-rpms-fc20-x86_64_merged/171/ : FAILURE
http://jenkins.ovirt.org/job/vdsm_master_unit-tests_merged/4164/ : FAILURE
vdsm-patches@lists.fedorahosted.org