LuaJIT - an alternative for current Python C bindings

[drago01]

On Thu, Nov 15, 2012 at 12:18 PM, Florian Weimer <fweimer at redhat.com> wrote:
> On 11/15/2012 01:49 AM, Alek Paunov wrote:
>
>> So, to me it seems natural joining all above together to start thinking
>> for replacing the classic python C bindings with thin textual or
>> bytecode(*) LuaJIT/FFI shims in benefit of things with great set of
>> dependencies like Anaconda (thanks to lupa, Lua objects behaves like
>> Python ones in Python). The result will be order of magnitude easier for
>> maintenance (towards the APIs evolution) and faster code.
>
>
> LuaJIT is currently not available for our secondary architectures, so this
> would need fixing first.  ("not available" meaning exactly that, there is no
> fallback to a slow C interpreter mode or something like that.)
>
> It's also not clear how significant the actual memory savings will be
> because some of the Lua OO frameworks have quite a bit of overhead.  And the
> eventually arriving abrt plugin will bring some weight with it, too.
>
> This is nothing against Lua or LuaJIT.  I just don't think that switching
> programming languages will magically reduce resource consumption.  This has
> to be an explicit design goal, or else it is not likely to happen.  I'm sure
> it's possible to write resource-conserving Python code, too.  It's just the
> baseline overhead from the run-time system which is unavoidable, but that's
> fairly small for Python (apparently less than 2 MB of unshared RSS per
> process).

Porting yum to python3 might be an easier task ... from the python 3.3
changelog [1]:

"The storage of Unicode strings now depends on the highest codepoint
in the string:

    pure ASCII and Latin1 strings (U+0000-U+00FF) use 1 byte per codepoint;
    BMP strings (U+0000-U+FFFF) use 2 bytes per codepoint;
    non-BMP strings (U+10000-U+10FFFF) use 4 bytes per codepoint.

The net effect is that for most applications, memory usage of string
storage should decrease significantly - especially compared to former
wide unicode builds - as, in many cases, strings will be pure ASCII
even in international contexts (because many strings store non-human
language data, such as XML fragments, HTTP headers, JSON-encoded data,
etc.). We also hope that it will, for the same reasons, increase CPU
cache efficiency on non-trivial applications. The memory usage of
Python 3.3 is two to three times smaller than Python 3.2, and a little
bit better than Python 2.7, on a Django benchmark (see the PEP for
details)."

[1] http://docs.python.org/3/whatsnew/3.3.html