LuaJIT - an alternative for current Python C bindings (was: Re: raising warning flag on firewalld-default feature)

[Alek Paunov]

On 12.11.2012 21:34, Steve Grubb wrote:
>
> But the problem I see is a lot of libraries are wrapped by swig, which leaks
> memory like a sieve.  If swig didn't generate such leaky code, Python based
> daemons wouldn't be as scary.
>

IMHO, Python is one of the best ways to express management logic. As 
have stated elsewhere in the thread, from the footprint perspective, 
python-libs can be decomposed to something including much smaller 
python-libs-core. The remaining problem is the quality of part of the C 
bindings. Otherwise, it is great advantage for Fedora that huge part of 
the system tools are already polished in Python (my own experience is 
that the C code in many tools/libs shipped with Fedora is of relative 
poor quality, as already Miloslav explained).

David Malcolm already has some real results addressing the Python 
bindings issues trough static analysis based on his great 
gcc-python-plugin/libcpychecker work. In the future, this machinery can 
be used for bug free bindings generation too.

Another option for addressing this is ... to just get rid of part of 
them in their current form.

LuaJIT 2.0 is a reimplementation of Lua scripting language, which has 
build the reputation of one of the fastest compilers (comparable only 
with gcc) in the course of his 3 years! beta cycle (2.0 final release 
was announced early this week)

The size of luajit.so (interpreter + trace compiler + C parser for the 
FFI definitions) is 452088 bytes (on x86_64, for comparison sqlite3.so, 
which is a python requirement is 677672)

Unlike other FFI implementations LuaJIT FFI works compiling the 
functions calls/C structs access to native code. Another difference is 
that the foreign C definitions are presented in native C syntax:

http://luajit.org/ext_ffi_tutorial.html#zlib

On top of this kind of FFI, LuaJIT already have bunch of library 
bindings - as instance the kernel ABI wrapper:

https://github.com/justincormack/ljsyscall

For a long time already Fedora ships lunatic-python (loading and 
bridging the Lua interpreter as python module)

More recent and feature complete bridge is:

https://github.com/scoder/lupa

from the author of lxml module.

So, to me it seems natural joining all above together to start thinking 
for replacing the classic python C bindings with thin textual or 
bytecode(*) LuaJIT/FFI shims in benefit of things with great set of 
dependencies like Anaconda (thanks to lupa, Lua objects behaves like 
Python ones in Python). The result will be order of magnitude easier for 
maintenance (towards the APIs evolution) and faster code.

Furthermore, we can get LuaJIT for free, because it is full API/ABI 
compatible with current Fedora (PUC Rio) Lua 5.1, which is pulled by RPM 
anyway - we can just replace Lua with LuaJIT as RPM provider for F20.

Kind Regards,
Alek

(*) Unlike original Lua, LuaJIT bytecode is portable across architectures.