9:41 a.m.
Hello Piotr Kliczewski,
I'd like you to do a code review. Please visit
https://gerrit.ovirt.org/65274
to review the following change.
Change subject: api: Speed up schema loading
......................................................................
api: Speed up schema loading
Use yaml.CLoader for 13x speedup in vdsm schema loading.
Here is a comparison with other formats:
$ python bench.py
yaml: 1.053720 seconds per load
cyaml: 0.086472 seconds per load
json: 0.049521 seconds per load
pickle: 0.001516 seconds per load
$ python3 bench.py
yaml: 1.340607 seconds per load
cyaml: 0.100010 seconds per load
json: 0.026627 seconds per load
pickle: 0.002153 seconds per load
$ grep 'model name' /proc/cpuinfo | head -n1
model name : Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Clearly the best format is pickle, 700X times faster on python 2 and
638X times faster on python 3. However loading the schema in 100
milliseconds seems to be good enough for now.
The benchmark script:
https://bugzilla.redhat.com/attachment.cgi?id=1208141
Change-Id: I230fb88c279973f542f6e1cac17f43ff6f0d2f62
Bug-Url: https://bugzilla.redhat.com/1381899
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
Reviewed-on: https://gerrit.ovirt.org/65230
Reviewed-by: Simone Tiraboschi <stirabos(a)redhat.com>
Continuous-Integration: Jenkins CI
Reviewed-by: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>
Tested-by: Simone Tiraboschi <stirabos(a)redhat.com>
---
M lib/api/vdsmapi.py
1 file changed, 1 insertion(+), 1 deletion(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/74/65274/1
diff --git a/lib/api/vdsmapi.py b/lib/api/vdsmapi.py
index b340f73..635aedf 100755
--- a/lib/api/vdsmapi.py
+++ b/lib/api/vdsmapi.py
@@ -92,7 +92,7 @@
try:
for path in paths:
with open(path) as f:
- loaded_schema = yaml.load(f)
+ loaded_schema = yaml.load(f, Loader=yaml.CLoader)
types = loaded_schema.pop('types')
self._types.update(types)
--
To view, visit https://gerrit.ovirt.org/65274
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I230fb88c279973f542f6e1cac17f43ff6f0d2f62
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: ovirt-4.0
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>
9:41 a.m.
gerrit-hooks has posted comments on this change.
Change subject: api: Speed up schema loading
......................................................................
Patch Set 1:
* #1208141::Update tracker: OK
* #1381899::Update tracker: OK
* #65230::Update tracker: OK
* Check Bug-Url::OK
* Check Public Bug::#1381899::OK, public bug
* Check Product::#1208141::IGNORE, not relevant for classification: Red Hat
* Check Product::#1381899::OK, product: vdsm
* Check Product::#65230::IGNORE, not relevant for classification: Retired
* Check TM::#1381899::ERROR, wrong target milestone for stable branch, ovirt-4.1.0-alpha
should match ^.*4.0.*
* Check merged to previous::OK, change not open on any previous branch
--
To view, visit https://gerrit.ovirt.org/65274
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I230fb88c279973f542f6e1cac17f43ff6f0d2f62
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: ovirt-4.0
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>
Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org>
Gerrit-HasComments: No
9:44 a.m.
Nir Soffer has posted comments on this change.
Change subject: api: Speed up schema loading
......................................................................
Patch Set 1: Verified+1
Verified on master and by users:
- http://lists.ovirt.org/pipermail/users/2016-October/043349.html
- http://lists.ovirt.org/pipermail/users/2016-October/043354.html
--
To view, visit https://gerrit.ovirt.org/65274
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I230fb88c279973f542f6e1cac17f43ff6f0d2f62
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: ovirt-4.0
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com>
Gerrit-Reviewer: Francesco Romani <fromani(a)redhat.com>
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com>
Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>
Gerrit-Reviewer: Yaniv Bronhaim <ybronhei(a)redhat.com>
Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org>
Gerrit-HasComments: No
4:44 a.m.
Piotr Kliczewski has posted comments on this change.
Change subject: api: Speed up schema loading
......................................................................
Patch Set 1: Code-Review+1
--
To view, visit https://gerrit.ovirt.org/65274
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I230fb88c279973f542f6e1cac17f43ff6f0d2f62
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: ovirt-4.0
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com>
Gerrit-Reviewer: Francesco Romani <fromani(a)redhat.com>
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com>
Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>
Gerrit-Reviewer: Yaniv Bronhaim <ybronhei(a)redhat.com>
Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org>
Gerrit-HasComments: No
3:59 a.m.
Dan Kenigsberg has posted comments on this change.
Change subject: api: Speed up schema loading
......................................................................
Patch Set 1: Code-Review+2
--
To view, visit https://gerrit.ovirt.org/65274
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I230fb88c279973f542f6e1cac17f43ff6f0d2f62
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: ovirt-4.0
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com>
Gerrit-Reviewer: Francesco Romani <fromani(a)redhat.com>
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com>
Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>
Gerrit-Reviewer: Yaniv Bronhaim <ybronhei(a)redhat.com>
Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org>
Gerrit-HasComments: No
4:02 a.m.
Francesco Romani has posted comments on this change.
Change subject: api: Speed up schema loading
......................................................................
Patch Set 1: Code-Review+1
--
To view, visit https://gerrit.ovirt.org/65274
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I230fb88c279973f542f6e1cac17f43ff6f0d2f62
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: ovirt-4.0
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com>
Gerrit-Reviewer: Francesco Romani <fromani(a)redhat.com>
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com>
Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>
Gerrit-Reviewer: Yaniv Bronhaim <ybronhei(a)redhat.com>
Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org>
Gerrit-HasComments: No
4:03 a.m.
Francesco Romani has posted comments on this change.
Change subject: api: Speed up schema loading
......................................................................
Patch Set 1:
assuming yaml.CLoader is available in the package we already depend on - some packagers
may want to split into a subpackage.
--
To view, visit https://gerrit.ovirt.org/65274
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I230fb88c279973f542f6e1cac17f43ff6f0d2f62
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: ovirt-4.0
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com>
Gerrit-Reviewer: Francesco Romani <fromani(a)redhat.com>
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com>
Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>
Gerrit-Reviewer: Yaniv Bronhaim <ybronhei(a)redhat.com>
Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org>
Gerrit-HasComments: No
8:37 a.m.
Dan Kenigsberg has submitted this change and it was merged.
Change subject: api: Speed up schema loading
......................................................................
api: Speed up schema loading
Use yaml.CLoader for 13x speedup in vdsm schema loading.
Here is a comparison with other formats:
$ python bench.py
yaml: 1.053720 seconds per load
cyaml: 0.086472 seconds per load
json: 0.049521 seconds per load
pickle: 0.001516 seconds per load
$ python3 bench.py
yaml: 1.340607 seconds per load
cyaml: 0.100010 seconds per load
json: 0.026627 seconds per load
pickle: 0.002153 seconds per load
$ grep 'model name' /proc/cpuinfo | head -n1
model name : Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Clearly the best format is pickle, 700X times faster on python 2 and
638X times faster on python 3. However loading the schema in 100
milliseconds seems to be good enough for now.
The benchmark script:
https://bugzilla.redhat.com/attachment.cgi?id=1208141
Change-Id: I230fb88c279973f542f6e1cac17f43ff6f0d2f62
Bug-Url: https://bugzilla.redhat.com/1381899
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
Reviewed-on: https://gerrit.ovirt.org/65230
Reviewed-by: Simone Tiraboschi <stirabos(a)redhat.com>
Continuous-Integration: Jenkins CI
Reviewed-by: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>
Tested-by: Simone Tiraboschi <stirabos(a)redhat.com>
Reviewed-on: https://gerrit.ovirt.org/65274
Reviewed-by: Dan Kenigsberg <danken(a)redhat.com>
Reviewed-by: Francesco Romani <fromani(a)redhat.com>
---
M lib/api/vdsmapi.py
1 file changed, 1 insertion(+), 1 deletion(-)
Approvals:
Piotr Kliczewski: Looks good to me, but someone else must approve
Nir Soffer: Verified
Jenkins CI: Passed CI tests
Dan Kenigsberg: Looks good to me, approved
Francesco Romani: Looks good to me, but someone else must approve
--
To view, visit https://gerrit.ovirt.org/65274
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I230fb88c279973f542f6e1cac17f43ff6f0d2f62
Gerrit-PatchSet: 2
Gerrit-Project: vdsm
Gerrit-Branch: ovirt-4.0
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com>
Gerrit-Reviewer: Francesco Romani <fromani(a)redhat.com>
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com>
Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>
Gerrit-Reviewer: Yaniv Bronhaim <ybronhei(a)redhat.com>
Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org>
8:37 a.m.
gerrit-hooks has posted comments on this change.
Change subject: api: Speed up schema loading
......................................................................
Patch Set 2:
* Update Tracker::#1381899::OK, status: MERGED
* Set MODIFIED::bug 1381899::::#1381899::::IGNORE, not all related patches are closed,
check 65452
--
To view, visit https://gerrit.ovirt.org/65274
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I230fb88c279973f542f6e1cac17f43ff6f0d2f62
Gerrit-PatchSet: 2
Gerrit-Project: vdsm
Gerrit-Branch: ovirt-4.0
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com>
Gerrit-Reviewer: Francesco Romani <fromani(a)redhat.com>
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com>
Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>
Gerrit-Reviewer: Yaniv Bronhaim <ybronhei(a)redhat.com>
Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org>
Gerrit-HasComments: No
2783
days inactive
2788
days old
vdsm-patches@lists.fedorahosted.org
8 comments
5 participants
participants (5)
-
automation＠ovirt.org -
Dan Kenigsberg -
fromani＠redhat.com -
Nir Soffer -
Piotr Kliczewski