Change in vdsm[master]: HACK: run GC in domain monitor

Adam Litke has uploaded a new change for review.

Change subject: HACK: run GC in domain monitor
......................................................................

HACK: run GC in domain monitor

Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Signed-off-by: Adam Litke <alitke(a)redhat.com&gt;
---
M vdsm/storage/monitor.py
M vdsm/storage/sdm/__init__.py
2 files changed, 22 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/80/40380/1

diff --git a/vdsm/storage/monitor.py b/vdsm/storage/monitor.py
index ef032a5..a2a1bd7 100644
--- a/vdsm/storage/monitor.py
+++ b/vdsm/storage/monitor.py
@@ -28,6 +28,7 @@
 
 from . import clusterlock
 from . import misc
+from . import sdm
 from .sdc import sdCache
 
 
@@ -249,6 +250,7 @@
             self._performDomainSelftest()
             self._checkReadDelay()
             self._collectStatistics()
+            self._garbageCollect()
         except Exception as e:
             self.log.exception("Error monitoring domain %s", self.sdUUID)
             self.nextStatus.error = e
@@ -340,6 +342,14 @@
         self.nextStatus.isoPrefix = self.isoPrefix
         self.nextStatus.version = self.domain.getVersion()
 
+    def _garbageCollect(self):
+        if True:  # XXX: limit this to domain ver 4 or later
+            try:
+                sdm.garbageCollectStorageDomain(self.domain)
+            except:
+                self.log.exception("Garbage collection failed for domain %s",
+                                   self.domain.sdUUID)
+
     # Managing host id
 
     def _shouldAcquireHostId(self):
diff --git a/vdsm/storage/sdm/__init__.py b/vdsm/storage/sdm/__init__.py
index 1636e63..4a31332 100644
--- a/vdsm/storage/sdm/__init__.py
+++ b/vdsm/storage/sdm/__init__.py
@@ -226,3 +226,15 @@
             cls.isolateVolumes(domain, srcImgUUID, dstImgUUID, volumeList)
     finally:
         domain.releaseClusterLock()
+
+
+def garbageCollectStorageDomain(domain):
+    if domain.isISO():
+        return
+    cls = __getStoreClass(domain)
+    hostId = getDomainHostId(domain.sdUUID)
+    domain.acquireClusterLock(hostId)
+    try:
+        cls.garbageCollectStorageDomain(domain)
+    finally:
+        domain.releaseClusterLock()


-- 
To view, visit https://gerrit.ovirt.org/40380
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com&gt;

[Jenkins CI RO]

oVirt Jenkins CI Server has posted comments on this change.

Change subject: HACK: run GC in domain monitor
......................................................................


Patch Set 1:

Build Started (1/2) -> http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/18316/

-- 
To view, visit https://gerrit.ovirt.org/40380
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com&gt;
Gerrit-Reviewer: oVirt Jenkins CI Server
Gerrit-HasComments: No

automation(a)ovirt.org has posted comments on this change.

Change subject: HACK: run GC in domain monitor
......................................................................


Patch Set 1:

* Update tracker::IGNORE, no Bug-Url found
* Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' and
is a valid url.
* Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.5',
'ovirt-3.4', 'ovirt-3.3'])

-- 
To view, visit https://gerrit.ovirt.org/40380
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com&gt;
Gerrit-Reviewer: automation(a)ovirt.org
Gerrit-Reviewer: oVirt Jenkins CI Server
Gerrit-HasComments: No

[Jenkins CI RO]

oVirt Jenkins CI Server has posted comments on this change.

Change subject: HACK: run GC in domain monitor
......................................................................


Patch Set 1:

Build Started (2/2) ->
http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/1546/

-- 
To view, visit https://gerrit.ovirt.org/40380
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com&gt;
Gerrit-Reviewer: automation(a)ovirt.org
Gerrit-Reviewer: oVirt Jenkins CI Server
Gerrit-HasComments: No

[Jenkins CI RO]

oVirt Jenkins CI Server has posted comments on this change.

Change subject: HACK: run GC in domain monitor
......................................................................


Patch Set 1: Verified-1

Build Failed 

http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/18316/ : SUCCESS

http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/1546/ : FAILURE

-- 
To view, visit https://gerrit.ovirt.org/40380
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com&gt;
Gerrit-Reviewer: automation(a)ovirt.org
Gerrit-Reviewer: oVirt Jenkins CI Server
Gerrit-HasComments: No

[Nir Soffer]

Nir Soffer has posted comments on this change.

Change subject: HACK: run GC in domain monitor
......................................................................


Patch Set 1:

(2 comments)

https://gerrit.ovirt.org/#/c/40380/1//COMMIT_MSG
Commit Message:

Line 4: Commit:     Adam Litke <alitke(a)redhat.com&gt;
Line 5: CommitDate: 2015-04-28 16:47:13 -0400
Line 6: 
Line 7: HACK: run GC in domain monitor
Line 8: 
I like this - this is not a hack! We have a thread dedicated to each storage domain, but
we do not use it for most storage domain operations.
Line 9: Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376


https://gerrit.ovirt.org/#/c/40380/1/vdsm/storage/monitor.py
File vdsm/storage/monitor.py:

Line 249: 
Line 250:             self._performDomainSelftest()
Line 251:             self._checkReadDelay()
Line 252:             self._collectStatistics()
Line 253:             self._garbageCollect()
The domain monitor must wake up every 10 seconds for monitoring the storage, so this must
be short operation that never blocks for long time, unless the blocking means that the
domain is not accessible.

Collecting garbage here means also that notifying state changes, acquiring host id and
setting next status will be delayed.

I think garbage collections should be integrated in the the monitor loop, after
_monitorDomain returns, something like this:

    deadline = monotonic_time()

    while True:
        deadline += interval
        monitor domain
        collect garbage before deadline
        sleep until deadline

So gc is performed in the time the monitor used to sleep before, and cannot disturb
reporting, unless the storage is stuck, which is a fine reason to disturb reporting.
Line 254:         except Exception as e:
Line 255:             self.log.exception("Error monitoring domain %s",
self.sdUUID)
Line 256:             self.nextStatus.error = e
Line 257: 


-- 
To view, visit https://gerrit.ovirt.org/40380
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com&gt;
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com&gt;
Gerrit-Reviewer: automation(a)ovirt.org
Gerrit-HasComments: Yes

automation(a)ovirt.org has posted comments on this change.

Change subject: HACK: run GC in domain monitor
......................................................................


Patch Set 2:

* Update tracker::IGNORE, no Bug-Url found
* Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' and
is a valid url.
* Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.5',
'ovirt-3.4', 'ovirt-3.3'])

-- 
To view, visit https://gerrit.ovirt.org/40380
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Gerrit-PatchSet: 2
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com&gt;
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com&gt;
Gerrit-Reviewer: automation(a)ovirt.org
Gerrit-HasComments: No

automation(a)ovirt.org has posted comments on this change.

Change subject: HACK: run GC in domain monitor
......................................................................


Patch Set 3:

* Update tracker::IGNORE, no Bug-Url found
* Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' and
is a valid url.
* Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.5',
'ovirt-3.4', 'ovirt-3.3'])

-- 
To view, visit https://gerrit.ovirt.org/40380
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Gerrit-PatchSet: 3
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com&gt;
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com&gt;
Gerrit-Reviewer: automation(a)ovirt.org
Gerrit-HasComments: No

[Jenkins CI RO]

Jenkins CI RO has abandoned this change.

Change subject: HACK: run GC in domain monitor
......................................................................


Abandoned

Abandoned due to no activity - please restore if still relevant

-- 
To view, visit https://gerrit.ovirt.org/40380
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Gerrit-PatchSet: 3
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com&gt;
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Jenkins CI RO
Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com&gt;
Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org&gt;

gerrit-hooks has posted comments on this change.

Change subject: HACK: run GC in domain monitor
......................................................................


Patch Set 3:

* Update tracker: IGNORE, no Bug-Url found

-- 
To view, visit https://gerrit.ovirt.org/40380
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Gerrit-PatchSet: 3
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com&gt;
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Jenkins CI RO
Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com&gt;
Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org&gt;
Gerrit-HasComments: No