[Fedora-suds-list] Caching Enhancements

[Matt C]

Great news, thanks Jeff. I'll start testing this later in the week and will
report back.
-Matt

On Wed, Jan 13, 2010 at 9:20 AM, Jeff Ortel <jortel at redhat.com> wrote:

> All,
>
> I finished and (committed r637) enhancements to the suds caching.  The
> previous approach added caching to the Transport layer and was causing some
> problems.  It /blindly/ cached downloaded XML documents such as WSDLs and
> imported XSD files as XML text.  The next time the document is needed (at
> the same URL), the transport returned the local (cached) copy of the XML
> file and it was SAX parsed as if it was downloaded via http.  This actually
> saved a lot of time because most servers returned these documents very
> slowly is many cases.  The problems with this approach are:
>  - Corrupt, incomplete or otherwise invalid XML document text was being
>    cached because the Transport layer was not content aware.  It simply
> cached
>    whatever it downloaded.
>  - Invalid cache files caused SAX parsing exceptions until the cache file
>    expired or the user cleared the cache.
>  - Cached files had to be SAX parsed every time the were retrieved.
>
> The approach taken by the r637 enhancement puts caching above the Transport
> layer and at the Document Processing layer.  By doing this, suds is now
> caching sax.document.Document objects using python pickle.  This ensures
> that cached XML documents are valid and allows for proper handling of
> corrept, incomplete or otherwise invalid cached files.  In the event that a
> cached file fails to load, the document processing layer simply uses the
> transport to download a good copy which is then SAX parsed and cached.
>
> I benchmarked the performance difference between sax parsing and unpickling
> a 2M XML file using cPickle.  In most cases, I observed a ~2x performance
> gain using pickle.load() over sax parsing.  Yeah, I'm a little surprised
> too.  I expected about a ~10x gain but at least it's significantly better.
>
> I tried really hard to preserve backwards compatibility but some minor
> things did have to change.  These things should not affect most users:
>  - The caching classes (Cache, FileCache, NoCache) moved from the
> suds.transport package
>    to a suds.cache.py module.
>  - All caching has been removed from the Transport classes.
>  - The get() & put() methods on the Cache interface have changed slightly
> to be
>    document oriented instead of file pointer (fp) or file-like oriented.
>  - Users disabling the cache using /NoCache/ should switch to the already
>    existing:  Client(url, cache=None) syntax.  Or, import NoCache from the
> suds.cache.
>  - Users implementing their own caches need to adapt to the changes to the
>    Cache interface.
>
> I plan to continue toward caching the entire WSDL (wsdl.Definitions) object
> but this is a good interim step and solves most of the caching problems for
> now.
>
> This has been released in 0.3.9 beta which I believe to be very stable.
>
> Hope this helps.
>
> Regards,
>
> Jeff
>
>
> _______________________________________________
> suds mailing list
> suds at lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/suds
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.fedoraproject.org/pipermail/suds/attachments/20100113/0a458f4a/attachment-0001.html