Change in vdsm[ovirt-3.5]: utils: Add memoized invalidation

Monday, 20 April 2015

Hello Piotr Kliczewski, Nir Soffer, Dan Kenigsberg,

I'd like you to do a code review.  Please visit

    https://gerrit.ovirt.org/40029

to review the following change.

Change subject: utils: Add memoized invalidation
......................................................................

utils: Add memoized invalidation

We learned in the hard way that memoizing and monkey-patching do work
nicely together. However, disabling memoizing during the tests means
that we do not test the same code in the application.

This patch adds invalidate() method to memoized functions, so tests can
clean up properly after they modify memozied data.

This can also be used by application code to invalidate memoized
functions in certain conditions.

Change-Id: I3053b8ecc567f7c3154f7473dfd6b587823313ae
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com&gt;
Reviewed-on: http://gerrit.ovirt.org/37788
Reviewed-by: Francesco Romani <fromani(a)redhat.com&gt;
Reviewed-by: Piotr Kliczewski <piotr.kliczewski(a)gmail.com&gt;
Reviewed-by: Dan Kenigsberg <danken(a)redhat.com&gt;
---
M lib/vdsm/utils.py
M tests/utilsTests.py
2 files changed, 67 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/29/40029/1

diff --git a/lib/vdsm/utils.py b/lib/vdsm/utils.py
index 4619b74..944cf4d 100644
--- a/lib/vdsm/utils.py
+++ b/lib/vdsm/utils.py
@@ -971,9 +971,14 @@
             self.cache[args] = value
             return value
 
+    def invalidate(self):
+        self.cache.clear()
+
     def __get__(self, obj, objtype):
         """Support instance methods."""
-        return functools.partial(self.__call__, obj)
+        wrapper = functools.partial(self.__call__, obj)
+        wrapper.invalidate = self.cache.clear
+        return wrapper
 
 
 def validateMinimalKeySet(dictionary, reqParams):
diff --git a/tests/utilsTests.py b/tests/utilsTests.py
index 5b3614d..2a0b8ef 100644
--- a/tests/utilsTests.py
+++ b/tests/utilsTests.py
@@ -18,6 +18,7 @@
 # Refer to the README and COPYING files for full details of the license
 #
 
+import collections
 import os.path
 import contextlib
 import errno
@@ -662,3 +663,63 @@
         self.assertTrue(
             hack < base/2,
             "picklecopy [%f] not faster than deepcopy [%f]" % (hack, base))
+
+
+@expandPermutations
+class MemoizedTests(TestCaseBase):
+
+    def setUp(self):
+        self.values = {}
+        self.accessed = collections.defaultdict(int)
+
+    @permutations([[()], [("a",)], [("a", "b")]])
+    def test_memoized_method(self, args):
+        self.values[args] = 42
+        self.assertEqual(self.accessed[args], 0)
+        self.assertEqual(self.memoized_method(*args), 42)
+        self.assertEqual(self.accessed[args], 1)
+        self.assertEqual(self.memoized_method(*args), 42)
+        self.assertEqual(self.accessed[args], 1)
+
+    @permutations([[()], [("a",)], [("a", "b")]])
+    def test_memoized_function(self, args):
+        self.values[args] = 42
+        self.assertEqual(self.accessed[args], 0)
+        self.assertEqual(memoized_function(self, *args), 42)
+        self.assertEqual(self.accessed[args], 1)
+        self.assertEqual(memoized_function(self, *args), 42)
+        self.assertEqual(self.accessed[args], 1)
+
+    def test_key_error(self):
+        self.assertRaises(KeyError, self.memoized_method)
+        self.assertRaises(KeyError, self.memoized_method, "a")
+        self.assertRaises(KeyError, self.memoized_method, "a", "b")
+
+    def test_invalidate_method(self):
+        args = ("a",)
+        self.values[args] = 42
+        self.assertEqual(self.memoized_method(*args), 42)
+        self.memoized_method.invalidate()
+        self.assertEqual(self.memoized_method(*args), 42)
+        self.assertEqual(self.accessed[args], 2)
+
+    def test_invalidate_function(self):
+        args = ("a",)
+        self.values[args] = 42
+        self.assertEqual(memoized_function(self, *args), 42)
+        memoized_function.invalidate()
+        self.assertEqual(memoized_function(self, *args), 42)
+        self.assertEqual(self.accessed[args], 2)
+
+    @utils.memoized
+    def memoized_method(self, *args):
+        return self.get(args)
+
+    def get(self, key):
+        self.accessed[key] += 1
+        return self.values[key]
+
+
+(a)utils.memoized
+def memoized_function(test, *args):
+    return test.get(args)


-- 
To view, visit https://gerrit.ovirt.org/40029
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I3053b8ecc567f7c3154f7473dfd6b587823313ae
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: ovirt-3.5
Gerrit-Owner: Francesco Romani <fromani(a)redhat.com&gt;
Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com&gt;
Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com&gt;
Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski(a)gmail.com&gt;

    

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011