Ok, recently Scott Henson, John Eckersberg, and I were trying to speed up cobbler ... a lot. We got a lot of good things done, including cutting the sync time in half. One thing we also achieved is we found out, that using the python profiler, even though the new yaml implementation is more robust (good, that's the most important thing!), it's also slow... even with libyaml installed. That all being said, the old implementation was also slow. We want /fast/.
Some quick benchmarks on my end show that saving objects using simplejson instead of PyYAML result in 10-12 second times to edit 200 objects versus 28 seconds.
As a result of this, I'm going to go ahead and add "serializer_catalog2" to the distribution, though I should warn you ... there's no upgrade path.
The way to replicate from catalog to catalog2 is to edit modules.conf on a /new/ cobbler machine and cobbler replicate the data to it (since the wire protocol is exactl y the same).
Simplejson offers two output modes -- one is a "pretty printed" mode, this mode is maybe 30% slower. It depends how cleanly readable we need the output to be, but the answer is "not really", since we can always get a pretty printed version of a data structure someone pastebins using ipython if absolutely needed. I am in favor of not using the pretty printed version.
Just for kicks, I think I'll benchmark cPickle again -- shelve (which uses things like berkley DB) tends to have issues.
We may make the JSON implementation the new default in 1.8, we'll see.
--Michael
Michael DeHaan wrote:
Ok, recently Scott Henson, John Eckersberg, and I were trying to speed up cobbler ... a lot. We got a lot of good things done, including cutting the sync time in half. One thing we also achieved is we found out, that using the python profiler, even though the new yaml implementation is more robust (good, that's the most important thing!), it's also slow... even with libyaml installed. That all being said, the old implementation was also slow. We want /fast/.
Some quick benchmarks on my end show that saving objects using simplejson instead of PyYAML result in 10-12 second times to edit 200 objects versus 28 seconds.
As a result of this, I'm going to go ahead and add "serializer_catalog2" to the distribution, though I should warn you ... there's no upgrade path.
The way to replicate from catalog to catalog2 is to edit modules.conf on a /new/ cobbler machine and cobbler replicate the data to it (since the wire protocol is exactl y the same).
Simplejson offers two output modes -- one is a "pretty printed" mode, this mode is maybe 30% slower. It depends how cleanly readable we need the output to be, but the answer is "not really", since we can always get a pretty printed version of a data structure someone pastebins using ipython if absolutely needed. I am in favor of not using the pretty printed version.
Just for kicks, I think I'll benchmark cPickle again -- shelve (which uses things like berkley DB) tends to have issues.
We may make the JSON implementation the new default in 1.8, we'll see.
--Michael _______________________________________________ cobbler mailing list cobbler@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/cobbler
Further data... it appears, and I don't really have the data to back this up just yet, that with, say 3000 systems in cobbler's configuration, "cobbler list" and "cobbler report" on these systems are basically instantaneous.
sync took 30 seconds with 3000 systems (using hardlinks), where before it could take 15 seconds with 300. Roughly, this seems like it's infinitely faster for loading. Almost seems too good to be true.
Writing is maybe 2x fast.
It would be trivial to write a seperate (not shipped) python script for migrating folks, if this proves as fast as I think it is in further tests.
The last time we tried this it was fed very large files (catalog style) -- now with the smaller ones it seems very very happy.
--Michael
On Tue, Mar 24, 2009 at 4:11 PM, Michael DeHaan mdehaan@redhat.com wrote:
Michael DeHaan wrote:
Ok, recently Scott Henson, John Eckersberg, and I were trying to speed up cobbler ... a lot. We got a lot of good things done, including cutting the sync time in half. One thing we also achieved is we found out, that using the python profiler, even though the new yaml implementation is more robust (good, that's the most important thing!), it's also slow... even with libyaml installed. That all being said, the old implementation was also slow. We want /fast/.
Some quick benchmarks on my end show that saving objects using simplejson instead of PyYAML result in 10-12 second times to edit 200 objects versus 28 seconds.
As a result of this, I'm going to go ahead and add "serializer_catalog2" to the distribution, though I should warn you ... there's no upgrade path.
The way to replicate from catalog to catalog2 is to edit modules.conf on a /new/ cobbler machine and cobbler replicate the data to it (since the wire protocol is exactl y the same).
Simplejson offers two output modes -- one is a "pretty printed" mode, this mode is maybe 30% slower. It depends how cleanly readable we need the output to be, but the answer is "not really", since we can always get a pretty printed version of a data structure someone pastebins using ipython if absolutely needed. I am in favor of not using the pretty printed version.
Just for kicks, I think I'll benchmark cPickle again -- shelve (which uses things like berkley DB) tends to have issues.
We may make the JSON implementation the new default in 1.8, we'll see.
--Michael _______________________________________________ cobbler mailing list cobbler@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/cobbler
Further data... it appears, and I don't really have the data to back this up just yet, that with, say 3000 systems in cobbler's configuration, "cobbler list" and "cobbler report" on these systems are basically instantaneous.
sync took 30 seconds with 3000 systems (using hardlinks), where before it could take 15 seconds with 300. Roughly, this seems like it's infinitely faster for loading. Almost seems too good to be true.
Writing is maybe 2x fast.
It would be trivial to write a seperate (not shipped) python script for migrating folks, if this proves as fast as I think it is in further tests.
The last time we tried this it was fed very large files (catalog style) -- now with the smaller ones it seems very very happy.
And this is why folks like us using cobbler in moderate to large environments love how fast you release. Great work Michael and crew.
Jeff Schroeder wrote:
On Tue, Mar 24, 2009 at 4:11 PM, Michael DeHaan mdehaan@redhat.com wrote:
Michael DeHaan wrote:
Ok, recently Scott Henson, John Eckersberg, and I were trying to speed up cobbler ... a lot. We got a lot of good things done, including cutting the sync time in half. One thing we also achieved is we found out, that using the python profiler, even though the new yaml implementation is more robust (good, that's the most important thing!), it's also slow... even with libyaml installed. That all being said, the old implementation was also slow. We want /fast/.
Some quick benchmarks on my end show that saving objects using simplejson instead of PyYAML result in 10-12 second times to edit 200 objects versus 28 seconds.
As a result of this, I'm going to go ahead and add "serializer_catalog2" to the distribution, though I should warn you ... there's no upgrade path.
The way to replicate from catalog to catalog2 is to edit modules.conf on a /new/ cobbler machine and cobbler replicate the data to it (since the wire protocol is exactl y the same).
Simplejson offers two output modes -- one is a "pretty printed" mode, this mode is maybe 30% slower. It depends how cleanly readable we need the output to be, but the answer is "not really", since we can always get a pretty printed version of a data structure someone pastebins using ipython if absolutely needed. I am in favor of not using the pretty printed version.
Just for kicks, I think I'll benchmark cPickle again -- shelve (which uses things like berkley DB) tends to have issues.
We may make the JSON implementation the new default in 1.8, we'll see.
--Michael _______________________________________________ cobbler mailing list cobbler@lists.fedorahosted.org https://fedorahosted.org/mailman/listinfo/cobbler
Further data... it appears, and I don't really have the data to back this up just yet, that with, say 3000 systems in cobbler's configuration, "cobbler list" and "cobbler report" on these systems are basically instantaneous.
sync took 30 seconds with 3000 systems (using hardlinks), where before it could take 15 seconds with 300. Roughly, this seems like it's infinitely faster for loading. Almost seems too good to be true.
Writing is maybe 2x fast.
It would be trivial to write a seperate (not shipped) python script for migrating folks, if this proves as fast as I think it is in further tests.
The last time we tried this it was fed very large files (catalog style) -- now with the smaller ones it seems very very happy.
And this is why folks like us using cobbler in moderate to large environments love how fast you release. Great work Michael and crew.
FYI -- I've decided to just teach catalog to update folks automatically, saving new files with an extension showing the type.
I can do this seamlessly, so why not :)
--Michael
Nice speedup stats.
New serializer (that automatically updates someone from YAML to JSON), creating 30,000 (yes, really) systems from scratch on the master branch:
120 seconds.
This is roughly150 times faster.
You wouldn't use DHCP management with that config until we implement network objects (1.8) that know about this and replicated servers and such, but I would just about declare any performance problems completely over.
The web app remains completely usable.
I can't explain the level of happiness this brings me -- a parser that doesn't care about indenting, newlines, and different quoting rules has a lot more freedom to be zippy.
Look forward to the speedups in 1.6. The YAML implementation will still be used for the main config file just to avoid upgrade pain, but that's it.
--Michael
cobbler@lists.fedorahosted.org
-
Jeff Schroeder -
Michael DeHaan