future of python3...
dmalcolm at redhat.com
Tue May 3 23:25:49 UTC 2011
On Wed, 2011-05-04 at 03:57 +0530, Rahul Sundaram wrote:
> On 05/03/2011 11:47 PM, David Malcolm wrote:
> > I'm also working on this Fedora 16 idea, which should help both PyPy and
> > CPython more robust:
> > http://fedoraproject.org/wiki/Features/StaticAnalysisOfCPythonExtensions
> > though it's early days yet, and I don't yet know how good the
> > signal:noise ratio for the tool will be, which will affect how tightly
> > we will want to integrate this into Fedora's build system. (I'm about
> > to go on vacation, and I don't want to open up the code until I get
> > back, later this month).
> > Hope this all sounds sane
> Yes indeed and thanks for the info. http://speed.pypy.org/ looks very
> impressive and with the tradeoff of less debugability, I wasn't sure
> how much it impacts the speed of say yum. Have you had a chance to look
> at that?
I haven't specifically looked at yum yet, but yum is heavily enmeshed
with the rpm python bindings, which are largely in C
Note that PyPy has to do extra work for C extension modules: the
PyObject* it provides are all just wrappers around PyPy's "real"
objects, so there is a CPU cost at the python/C boundary that isn't
present in CPython.
This, and the fact that the C code is invisible to the JIT compiler
means that a workload that's calling into C extension code a lot might
not gain much from the JIT compiler. (Optimization advice for PyPy is
often to recode C back into Python, since pure Python code is amenable
to optimizations when being JIT-compiled). So additional work may be
needed to support pypy running yum and to get speed and memory usage
improvements, and I don't know if it would be worth it.
So far, every time I've tried to use PyPy's C extension support, the
extension has either not built, or it's segfaulted on me very quickly.
This was with 1.4 and 1.4.1 though, and a lot of work was done in 1.5 on
If you want to play with this, you could try the pypy-devel rpm, or try
building pypy from source, but obviously you'll want to do it on a
testing rpm db, not on the rpm db of a machine you care about ;)
It's probably only worth doing if you're familiar with gdb, too, and are
willing to dive deep into the guts of PyPy, rpm and yum :) I may have a
chance to do this when I get back from vacation.
More information about the test