12:01 p.m.
Hi everyone,
I might be in a strange and careless mood today, but here is something I
wanted to suggest since the time I saw the amount of logic that goes into SSSD
and is implemented in C.
What if we implement some of the complicated logic inside SSSD in Lua [1]?
Lua is a language specifically designed to be embedded. It grew into
popularity after it was embedded into World Of Warcraft. It was also embedded
and is still being embedded into a bunch of other games. However, games is far
from the only application for Lua.
Lua is a simple and lightweight language, yet with lots of flexibility -
grasping the essence of the whole reference manual [2] takes one evening and
it's an entertaining read. The VM is small and fast. In fact some people use
it on microcontrollers [3].
I personally embedded it into two C-based projects, one of which was already
quite big, and another was a rewrite of a smallish, but complicated C project.
In both cases, where before things were tedious and hard, they have become
easy and fun. The first project was on an embedded platform. I also had some
other, but smaller stuff done with it.
Generally, Lua gets put into the middle with outside interfaces and
performance-critical parts implemented in C. With a very simple interface to
and from C one can put boundaries wherever necessary. Yes, the interface is
simpler than Python's.
The process is this: you implement some C libraries (shared/static) which are
loadable from Lua (Lua->C interface) and then create a VM or several, load
some Lua code and call into it (C->Lua interface).
Some of the benefits are:
Clearer logic (higher level language)
Much less memory leaks and very little memory management (mostly in Lua->C
interface).
Faster development cycle (no compilation, edit and retry on a live daemon)
Potentially easy plugin interface (yay for admin scripts!)
Some of the drawbacks are:
More difficult to do low level stuff (so we just don't do that)
Lower performance than C (still good enough for games, and we can have the
critical parts in C easily)
Needs learning and development investment.
Regarding the development investment, due to low cost of C/Lua interface
implementation, it is practical to replace the relevant C code piece-by-piece,
provided pieces are not too small (say, 2KLOC and up).
Lastly, don't take this too seriously. I realise that this is disruptive and
hard. However if you ever feel like SSSD has grown too big and complex, ping
me, or just try Lua yourself :)
Nick
[1] http://www.lua.org/
[2] http://www.lua.org/manual/5.3/manual.html
[3] http://www.eluaproject.net/
1:37 p.m.
On Fri, 2015-08-21 at 20:01 +0300, Nikolai Kondrashov wrote:
Hi everyone,
I might be in a strange and careless mood today, but here is something I
wanted to suggest since the time I saw the amount of logic that goes into SSSD
and is implemented in C.
What if we implement some of the complicated logic inside SSSD in Lua [1]?
Lua is a language specifically designed to be embedded. It grew into
popularity after it was embedded into World Of Warcraft. It was also embedded
and is still being embedded into a bunch of other games. However, games is far
from the only application for Lua.
Lua is a simple and lightweight language, yet with lots of flexibility -
grasping the essence of the whole reference manual [2] takes one evening and
it's an entertaining read. The VM is small and fast. In fact some people use
it on microcontrollers [3].
I personally embedded it into two C-based projects, one of which was already
quite big, and another was a rewrite of a smallish, but complicated C project.
In both cases, where before things were tedious and hard, they have become
easy and fun. The first project was on an embedded platform. I also had some
other, but smaller stuff done with it.
Generally, Lua gets put into the middle with outside interfaces and
performance-critical parts implemented in C. With a very simple interface to
and from C one can put boundaries wherever necessary. Yes, the interface is
simpler than Python's.
The process is this: you implement some C libraries (shared/static) which are
loadable from Lua (Lua->C interface) and then create a VM or several, load
some Lua code and call into it (C->Lua interface).
Some of the benefits are:
Clearer logic (higher level language)
Much less memory leaks and very little memory management (mostly in Lua->C
interface).
Faster development cycle (no compilation, edit and retry on a live daemon)
Potentially easy plugin interface (yay for admin scripts!)
Some of the drawbacks are:
More difficult to do low level stuff (so we just don't do that)
Lower performance than C (still good enough for games, and we can have the
critical parts in C easily)
Needs learning and development investment.
Regarding the development investment, due to low cost of C/Lua interface
implementation, it is practical to replace the relevant C code piece-by-piece,
provided pieces are not too small (say, 2KLOC and up).
Lastly, don't take this too seriously. I realise that this is disruptive and
hard. However if you ever feel like SSSD has grown too big and complex, ping
me, or just try Lua yourself :)
Nick
[1] http://www.lua.org/
[2] http://www.lua.org/manual/5.3/manual.html
[3] http://www.eluaproject.net/
Hi Nick,
can you provide an example of a piece of SSSD youd replace with Lua ?
I am not asking for an implementation but a high level view of what a
function looks like to day and what it would look like if we were going
to use Lua (ideally with a list of primitives we'd still need to
provide, to understand how much code Lua replaces for real.
Simo.
--
Simo Sorce * Red Hat, Inc * New York
3:54 a.m.
On 08/21/2015 07:01 PM, Nikolai Kondrashov wrote:
Hi everyone,
I might be in a strange and careless mood today, but here is something I
wanted to suggest since the time I saw the amount of logic that goes
into SSSD
and is implemented in C.
What if we implement some of the complicated logic inside SSSD in Lua [1]?
Lua is a language specifically designed to be embedded. It grew into
popularity after it was embedded into World Of Warcraft. It was also
embedded
and is still being embedded into a bunch of other games. However, games
is far
from the only application for Lua.
Lua is a simple and lightweight language, yet with lots of flexibility -
grasping the essence of the whole reference manual [2] takes one evening
and
it's an entertaining read. The VM is small and fast. In fact some people
use
it on microcontrollers [3].
I personally embedded it into two C-based projects, one of which was
already
quite big, and another was a rewrite of a smallish, but complicated C
project.
In both cases, where before things were tedious and hard, they have become
easy and fun. The first project was on an embedded platform. I also had
some
other, but smaller stuff done with it.
Generally, Lua gets put into the middle with outside interfaces and
performance-critical parts implemented in C. With a very simple
interface to
and from C one can put boundaries wherever necessary. Yes, the interface is
simpler than Python's.
The process is this: you implement some C libraries (shared/static)
which are
loadable from Lua (Lua->C interface) and then create a VM or several, load
some Lua code and call into it (C->Lua interface).
Some of the benefits are:
Clearer logic (higher level language)
Much less memory leaks and very little memory management (mostly in
Lua->C
interface).
Faster development cycle (no compilation, edit and retry on a live
daemon)
Potentially easy plugin interface (yay for admin scripts!)
Some of the drawbacks are:
More difficult to do low level stuff (so we just don't do that)
Lower performance than C (still good enough for games, and we can
have the
critical parts in C easily)
Needs learning and development investment.
Regarding the development investment, due to low cost of C/Lua interface
implementation, it is practical to replace the relevant C code
piece-by-piece,
provided pieces are not too small (say, 2KLOC and up).
Lastly, don't take this too seriously. I realise that this is disruptive
and
hard. However if you ever feel like SSSD has grown too big and complex,
ping
me, or just try Lua yourself :)
Nick
[1] http://www.lua.org/
[2] http://www.lua.org/manual/5.3/manual.html
[3] http://www.eluaproject.net/
Hi,
can you tell us what features of Lua do you like and might help simplify
SSSD? I don't know the language, just fast scrolled over the manual and
seen few examples. It seems to be similar to javascript.
9:06 a.m.
On 08/21/2015 07:01 PM, Nikolai Kondrashov wrote:
Hi everyone,
I might be in a strange and careless mood today, but here is something I
wanted to suggest since the time I saw the amount of logic that goes
into SSSD
and is implemented in C.
What if we implement some of the complicated logic inside SSSD in Lua [1]?
Lua is a language specifically designed to be embedded. It grew into
popularity after it was embedded into World Of Warcraft. It was also
embedded
and is still being embedded into a bunch of other games. However, games
is far
from the only application for Lua.
Lua is a simple and lightweight language, yet with lots of flexibility -
grasping the essence of the whole reference manual [2] takes one evening
and
it's an entertaining read. The VM is small and fast. In fact some people
use
it on microcontrollers [3].
I personally embedded it into two C-based projects, one of which was
already
quite big, and another was a rewrite of a smallish, but complicated C
project.
In both cases, where before things were tedious and hard, they have become
easy and fun. The first project was on an embedded platform. I also had
some
other, but smaller stuff done with it.
Generally, Lua gets put into the middle with outside interfaces and
performance-critical parts implemented in C. With a very simple
interface to
and from C one can put boundaries wherever necessary. Yes, the interface is
simpler than Python's.
The process is this: you implement some C libraries (shared/static)
which are
loadable from Lua (Lua->C interface) and then create a VM or several, load
some Lua code and call into it (C->Lua interface).
Some of the benefits are:
Clearer logic (higher level language)
Much less memory leaks and very little memory management (mostly in
Lua->C
interface).
Faster development cycle (no compilation, edit and retry on a live
daemon)
Potentially easy plugin interface (yay for admin scripts!)
Some of the drawbacks are:
More difficult to do low level stuff (so we just don't do that)
Lower performance than C (still good enough for games, and we can
have the
critical parts in C easily)
Needs learning and development investment.
Regarding the development investment, due to low cost of C/Lua interface
implementation, it is practical to replace the relevant C code
piece-by-piece,
provided pieces are not too small (say, 2KLOC and up).
Lastly, don't take this too seriously. I realise that this is disruptive
and
hard. However if you ever feel like SSSD has grown too big and complex,
ping
me, or just try Lua yourself :)
Nick
I may be too conservative here, but I believe SSSD should
be written in language with static type-checking at least
as strong as C.
Michal
7:57 a.m.
Hi Simo,
On 08/21/2015 09:37 PM, Simo Sorce wrote:
On Fri, 2015-08-21 at 20:01 +0300, Nikolai Kondrashov wrote:
> I might be in a strange and careless mood today, but here is something I
> wanted to suggest since the time I saw the amount of logic that goes into SSSD
> and is implemented in C.
>
> What if we implement some of the complicated logic inside SSSD in Lua [1]?
can you provide an example of a piece of SSSD youd replace with Lua ?
I am not asking for an implementation but a high level view of what a
function looks like to day and what it would look like if we were going
to use Lua (ideally with a list of primitives we'd still need to
provide, to understand how much code Lua replaces for real.
I don't have enough knowledge of SSSD to provide really meaningful examples.
I have to know developer pain points to really suggest an interesting
solution, and I have to really know the code to show good examples.
I don't have the time now to really dive into the code and give a good answer,
I'm sorry. If there's interest and support for this idea, I will be glad to
dig in, give you more, and help with coding and design if it comes to that.
Still, judging from the code distribution and overall architecture, I would
say the first candidates will be providers. They are very complicated and have
a relatively well defined interface to SSSD core. Most of them talk over
network and any slowdown from Lua will likely be swamped by network latency.
I would make a (partial?) adaptor from the current interface to Lua, allowing
implementing a (limited?) provider in Lua, plus low-level communication
libraries underneath. This will allow testing the feasibility without
disrupting the existing providers.
If the idea proves useful, existing providers can be re-implemented in Lua,
perhaps section-by-section, and starting from the smallest. If all the
providers are converted, the adaptor can be removed and the core will talk
directly to high-level Lua provider code. One side-effect of such work will
be refactoring and simplifying of the provider interface, which will have a
positive effect on the overall architecture.
I expect main savings to come from exception support, automatic memory
management, simpler data massaging, and brevity of a scripting language (e.g.
implicit, dynamic typing).
Nick
8:14 a.m.
Hi Pavel,
On 08/24/2015 11:54 AM, Pavel Březina wrote:
On 08/21/2015 07:01 PM, Nikolai Kondrashov wrote:
> I might be in a strange and careless mood today, but here is something I
> wanted to suggest since the time I saw the amount of logic that goes into
> SSSD and is implemented in C.
>
> What if we implement some of the complicated logic inside SSSD in Lua [1]?
can you tell us what features of Lua do you like and might help simplify
SSSD? I don't know the language, just fast scrolled over the manual and seen
few examples. It seems to be similar to javascript.
Most of all, I like its simplicity, then flexibility. And then how easy it is
to embed. The latter comes from simplicity, first of all.
I listed the features I think will help us in the other message in this
thread, but here they are again and then some:
* exception handling
(somewhat rudimentary, but workable)
* automatic memory management
(incremental mark-and-sweep with good controls, customizable allocator)
* easier data massaging
(string operations, built-in (limited) regexes, etc.)
* brevity
(implicit dynamic typing, syntax sugar)
* OOP support
(there are several libraries to choose from)
I don't know enough JavaScript to really compare, but to me knowing that it
has two pairs of equality operators [1] is quite off-putting.
There is, however, a seemingly good comparison, which should give the general
idea:
http://stackoverflow.com/a/1022683/1161045
Nick
[1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference#Equalit...
8:21 a.m.
Hi Michal,
On 08/24/2015 05:06 PM, Michal Židek wrote:
On 08/21/2015 07:01 PM, Nikolai Kondrashov wrote:
> I might be in a strange and careless mood today, but here is something I
> wanted to suggest since the time I saw the amount of logic that goes into
> SSSD and is implemented in C.
>
> What if we implement some of the complicated logic inside SSSD in Lua [1]?
I may be too conservative here, but I believe SSSD should be written in
language with static type-checking at least as strong as C.
I agree that C's static typing is very useful, even though C also requires
implicit typing bringing verbosity. Still, using Lua is a tradeoff. We can
trade static typing for the benefits Lua brings, which mostly boil down to
smaller, clearer code.
BTW, a while ago I read a good treatise on the topic, which I'm happy to
share:
https://cdsmith.wordpress.com/2011/01/09/an-old-article-i-wrote/
Nick
8:23 a.m.
On 08/25/2015 03:57 PM, Nikolai Kondrashov wrote:
I expect main savings to come from exception support, automatic
memory
management, simpler data massaging, and brevity of a scripting language (e.g.
implicit, dynamic typing).
Oh, and another thing which scripting languages bring and which I'm not tiring
of mentioning is live, hands-on development. We can make SSSD reload Lua code
without recompilation and restarting, speeding up
development/debugging/testing cycle.
Nick
9:20 a.m.
On 08/25/2015 04:21 PM, Nikolai Kondrashov wrote:
Hi Michal,
On 08/24/2015 05:06 PM, Michal Židek wrote:
> On 08/21/2015 07:01 PM, Nikolai Kondrashov wrote:
>> I might be in a strange and careless mood today, but here is something I
>> wanted to suggest since the time I saw the amount of logic that goes into
>> SSSD and is implemented in C.
>>
>> What if we implement some of the complicated logic inside SSSD in Lua [1]?
>
> I may be too conservative here, but I believe SSSD should be written in
> language with static type-checking at least as strong as C.
I agree that C's static typing is very useful, even though C also requires
implicit typing bringing verbosity.
I meant "explicit" here, of course.
Nick
10:05 a.m.
On 08/25/2015 03:21 PM, Nikolai Kondrashov wrote:
Hi Michal,
On 08/24/2015 05:06 PM, Michal Židek wrote:
> On 08/21/2015 07:01 PM, Nikolai Kondrashov wrote:
>> I might be in a strange and careless mood today, but here is something I
>> wanted to suggest since the time I saw the amount of logic that goes
>> into
>> SSSD and is implemented in C.
>>
>> What if we implement some of the complicated logic inside SSSD in Lua
>> [1]?
>
> I may be too conservative here, but I believe SSSD should be written in
> language with static type-checking at least as strong as C.
I agree that C's static typing is very useful, even though C also requires
implicit typing bringing verbosity. Still, using Lua is a tradeoff. We can
trade static typing for the benefits Lua brings, which mostly boil down to
smaller, clearer code.
BTW, a while ago I read a good treatise on the topic, which I'm happy to
share:
https://cdsmith.wordpress.com/2011/01/09/an-old-article-i-wrote/
Nick
Before we use Lua (or any additional language) we need to do some
major refactoring inside SSSD. Create internal libraries with clean
API that operates on defined levels of abstraction for every bigger
component in SSSD (like responders, providers, sysdb etc.) and than
re-implement these components using the language of choice and
these libraries (libraries would still be in C).
Even if the language of choice for the higher level components
remains to be C, such refactoring will IMO sooner or later be
needed. However it will be a *very* big task and we would need
to plan how to deliver such change into SSSD.
Internally in SSSD we are far from clean APIs. It partially
may be because we use C on all levels of abstractions which
sometimes leads to their unwanted merging. We sometimes use
structures from lower levels too high in the stack or mix the
parts in all directions to simply make stuff work somehow
and do not always think about clean design. Using more abstract
language for upper (or top) layers may help to fight this
issue a little, but the main issue is still bad API
of the internal libraries and their unclear position in the
stack. Once we fix these issues we can start thinking
about adopting new language into the project but the
possibility is that it will no longer be such an issue at
that point.
Anyway, I think you raised a good topic to think about.
Michal
10:44 a.m.
On 08/25/2015 06:05 PM, Michal Židek wrote:
On 08/25/2015 03:21 PM, Nikolai Kondrashov wrote:
> On 08/24/2015 05:06 PM, Michal Židek wrote:
>> On 08/21/2015 07:01 PM, Nikolai Kondrashov wrote:
>>> I might be in a strange and careless mood today, but here is something I
>>> wanted to suggest since the time I saw the amount of logic that goes
>>> into
>>> SSSD and is implemented in C.
>>>
>>> What if we implement some of the complicated logic inside SSSD in Lua
>>> [1]?
>>
>> I may be too conservative here, but I believe SSSD should be written in
>> language with static type-checking at least as strong as C.
>
> I agree that C's static typing is very useful, even though C also requires
> implicit typing bringing verbosity. Still, using Lua is a tradeoff. We can
> trade static typing for the benefits Lua brings, which mostly boil down to
> smaller, clearer code.
>
> BTW, a while ago I read a good treatise on the topic, which I'm happy to
> share:
>
> https://cdsmith.wordpress.com/2011/01/09/an-old-article-i-wrote/
Before we use Lua (or any additional language) we need to do some
major refactoring inside SSSD. Create internal libraries with clean
API that operates on defined levels of abstraction for every bigger
component in SSSD (like responders, providers, sysdb etc.) and than
re-implement these components using the language of choice and
these libraries (libraries would still be in C).
Even if the language of choice for the higher level components
remains to be C, such refactoring will IMO sooner or later be
needed. However it will be a *very* big task and we would need
to plan how to deliver such change into SSSD.
Internally in SSSD we are far from clean APIs. It partially
may be because we use C on all levels of abstractions which
sometimes leads to their unwanted merging. We sometimes use
structures from lower levels too high in the stack or mix the
parts in all directions to simply make stuff work somehow
and do not always think about clean design. Using more abstract
language for upper (or top) layers may help to fight this
issue a little, but the main issue is still bad API
of the internal libraries and their unclear position in the
stack. Once we fix these issues we can start thinking
about adopting new language into the project but the
possibility is that it will no longer be such an issue at
that point.
Anyway, I think you raised a good topic to think about.
Thank you, Michal. I agree that sssd is very much in need of refactoring and
once we'll have it done, the code will be easier to read and understand.
However, it will still be C, with all the associated overhead for the high
level code. Also, it will stay C. Refactoring never ends, it's like weeding
out a garden. So, we'll have to refactor C again and again.
If the target of refactoring is written in a more suitable language, it will
be that bit more pleasant and effective, and subsequent refactoring will be
easier as well.
Although, I suspect once/if we get a more powerful tool, we'll simply use it
to build a more complex system and will be back to square one, but at least
we'll get more features done :)
Nick
10:59 a.m.
On (25/08/15 15:57), Nikolai Kondrashov wrote:
Hi Simo,
On 08/21/2015 09:37 PM, Simo Sorce wrote:
>On Fri, 2015-08-21 at 20:01 +0300, Nikolai Kondrashov wrote:
>>I might be in a strange and careless mood today, but here is something I
>>wanted to suggest since the time I saw the amount of logic that goes into SSSD
>>and is implemented in C.
>>
>>What if we implement some of the complicated logic inside SSSD in Lua [1]?
>
>can you provide an example of a piece of SSSD youd replace with Lua ?
>I am not asking for an implementation but a high level view of what a
>function looks like to day and what it would look like if we were going
>to use Lua (ideally with a list of primitives we'd still need to
>provide, to understand how much code Lua replaces for real.
I don't have enough knowledge of SSSD to provide really meaningful examples.
I have to know developer pain points to really suggest an interesting
solution, and I have to really know the code to show good examples.
SSSD is written in asynchronous way and tightly coupled with tevent[1] and talloc.
This is not use case for using scripting language. The ideal use case for scripting
languages is to write higly optimized library in C + bindings. The scipting language
would be used for replacing boilerplate code in C.
However there are still few ways.
You can write simple example of using lua scripts with tevent[1].
Then we can evaluate pros and cons of using Lua.
The another way would be to replace tevent[1] with lua implementation of event loop.
In another words, rewrite sssd from scratch.
Still, judging from the code distribution and overall architecture, I
would
say the first candidates will be providers. They are very complicated and have
a relatively well defined interface to SSSD core. Most of them talk over
network and any slowdown from Lua will likely be swamped by network latency.
Providers are very complicated due to compicated servers (LDAP, IPA, AD)
We will not be able to replace high level iplementation with scripting language
in short time (Lua -> C) . In short time, there still will need to be two way
comunication between C and Lua (Lua <-> C). Which would not be simplify code as
the first way.
I would make a (partial?) adaptor from the current interface to Lua,
allowing
implementing a (limited?) provider in Lua, plus low-level communication
libraries underneath. This will allow testing the feasibility without
disrupting the existing providers.
Will it be asynchronous?
In theory, you could use threads from scripting language but
we there could be synchronisation related issues. So the ideal would be to use
Erlang and message passing instead of shared variables.
If the idea proves useful, existing providers can be re-implemented in
Lua,
perhaps section-by-section, and starting from the smallest. If all the
providers are converted, the adaptor can be removed and the core will talk
directly to high-level Lua provider code. One side-effect of such work will
be refactoring and simplifying of the provider interface, which will have a
positive effect on the overall architecture.
There is a missing step between a POC and re-implementing existing providers.
We need to write functional test to assure we will not create a regressions.
As I already wrote servers are complicated (LDAP, IPA, AD); So code coverage
will need to be very high.
I expect main savings to come from
exception support,
scripting language might help
automatic memory management,
talloc helps a lot even in C.
simpler data massaging,
What do you mean here?
and brevity of a scripting language (e.g. implicit, dynamic typing).
This can be pros and cons in the same time.
Summary:
This is an interesting idea; but currently I cannot see many benefits
for using any scripting language in SSSD. Moreover sssd is a security
service. Such step could cause problems with security certifications.
LS
[1] https://tevent.samba.org/
11:04 a.m.
On 08/25/2015 04:14 PM, Nikolai Kondrashov wrote:
Hi Pavel,
On 08/24/2015 11:54 AM, Pavel Březina wrote:
> On 08/21/2015 07:01 PM, Nikolai Kondrashov wrote:
>> I might be in a strange and careless mood today, but here is something I
>> wanted to suggest since the time I saw the amount of logic that goes into
>> SSSD and is implemented in C.
>>
>> What if we implement some of the complicated logic inside SSSD in Lua [1]?
>
> can you tell us what features of Lua do you like and might help simplify
> SSSD? I don't know the language, just fast scrolled over the manual and seen
> few examples. It seems to be similar to javascript.
Most of all, I like its simplicity, then flexibility. And then how easy it is
to embed. The latter comes from simplicity, first of all.
I listed the features I think will help us in the other message in this
thread, but here they are again and then some:
* exception handling
(somewhat rudimentary, but workable)
* automatic memory management
(incremental mark-and-sweep with good controls, customizable allocator)
* easier data massaging
(string operations, built-in (limited) regexes, etc.)
* brevity
(implicit dynamic typing, syntax sugar)
* OOP support
(there are several libraries to choose from)
I don't know enough JavaScript to really compare, but to me knowing that it
has two pairs of equality operators [1] is quite off-putting.
There is, however, a seemingly good comparison, which should give the general
idea:
http://stackoverflow.com/a/1022683/1161045
Some things about that comparison:
* Lua supports bitwise operators now.
* There are no typeof and instanceof operators, but functions doing that
are usually implemented in OOP libraries
* It doesn't mention that Lua uses 1-based array indices by default [1].
It feels awkward at first, but you can get used to it.
Nick
[1] A long discussion on the official wiki :)
http://lua-users.org/wiki/CountingFromOne
11:45 a.m.
On 08/25/2015 06:59 PM, Lukas Slebodnik wrote:
On (25/08/15 15:57), Nikolai Kondrashov wrote:
> Hi Simo,
>
> On 08/21/2015 09:37 PM, Simo Sorce wrote:
>> On Fri, 2015-08-21 at 20:01 +0300, Nikolai Kondrashov wrote:
>>> I might be in a strange and careless mood today, but here is something I
>>> wanted to suggest since the time I saw the amount of logic that goes into
SSSD
>>> and is implemented in C.
>>>
>>> What if we implement some of the complicated logic inside SSSD in Lua [1]?
>>
>> can you provide an example of a piece of SSSD youd replace with Lua ?
>> I am not asking for an implementation but a high level view of what a
>> function looks like to day and what it would look like if we were going
>> to use Lua (ideally with a list of primitives we'd still need to
>> provide, to understand how much code Lua replaces for real.
>
> I don't have enough knowledge of SSSD to provide really meaningful examples.
> I have to know developer pain points to really suggest an interesting
> solution, and I have to really know the code to show good examples.
>
SSSD is written in asynchronous way and tightly coupled with tevent[1] and
talloc. This is not use case for using scripting language. The ideal use
case for scripting languages is to write higly optimized library in C +
bindings. The scipting language would be used for replacing boilerplate code
in C. However there are still few ways. You can write simple example of
using lua scripts with tevent[1]. Then we can evaluate pros and cons of
using Lua.
Now we're talking specifics, awesome! Alright, perhaps I'll do that. Care to
give a use case I should try to implement?
The another way would be to replace tevent[1] with lua implementation
of event loop.
In another words, rewrite sssd from scratch.
Whoa, no way :)
> Still, judging from the code distribution and overall
architecture, I would
> say the first candidates will be providers. They are very complicated and have
> a relatively well defined interface to SSSD core. Most of them talk over
> network and any slowdown from Lua will likely be swamped by network latency.
>
Providers are very complicated due to compicated servers (LDAP, IPA, AD)
We will not be able to replace high level iplementation with scripting language
in short time (Lua -> C) . In short time, there still will need to be two way
comunication between C and Lua (Lua <-> C). Which would not be simplify code as
the first way.
Sure. It's complicated. That's why a higher level language might be
beneficial. If we can cut off some parts of a provider cleanly (which is a
good idea for refactoring anyway), we can reimplement them in Lua with
relatively low overhead. Thus, going by pieces we can reduce the disruption.
> I would make a (partial?) adaptor from the current interface to
Lua, allowing
> implementing a (limited?) provider in Lua, plus low-level communication
> libraries underneath. This will allow testing the feasibility without
> disrupting the existing providers.
Will it be asynchronous?
In theory, you could use threads from scripting language but
we there could be synchronisation related issues.
Well, we can try it? I assume a lot of people did such things already.
There is a bunch of multithreading libraries for Lua, and then there are
always the built-in coroutines.
So the ideal would be to use Erlang and message passing instead of
shared
variables.
Sadly, I don't know Erlang yet, can't say anything about that. I bet it's
awesome. However, I expect it is harder to interface with as well.
> If the idea proves useful, existing providers can be
re-implemented in Lua,
> perhaps section-by-section, and starting from the smallest. If all the
> providers are converted, the adaptor can be removed and the core will talk
> directly to high-level Lua provider code. One side-effect of such work will
> be refactoring and simplifying of the provider interface, which will have a
> positive effect on the overall architecture.
>
There is a missing step between a POC and re-implementing existing providers.
We need to write functional test to assure we will not create a regressions.
As I already wrote servers are complicated (LDAP, IPA, AD); So code coverage
will need to be very high.
Yeah, this bothers me too, it is dangerous, I agree. Perhaps if we do it
slowly, piece-by-piece, it will be safer and easier to manage, i.e. write
tests for, or test manually?
> I expect main savings to come from
> exception support,
scripting language might help
> automatic memory management,
talloc helps a lot even in C.
I agree. Still, it's explicit and needs taking care of, constantly.
> simpler data massaging,
What do you mean here?
The messy stuff. Parsing and gluing strings together, maintaining complicated
data structures, converting from one representation to another, etc.
> and brevity of a scripting language (e.g. implicit, dynamic
typing).
>
This can be pros and cons in the same time.
Implicit, dynamic typing - yes. Brevity - not likely. You can always make it
more verbose, after all, but it is hard the other way around.
Summary:
This is an interesting idea; but currently I cannot see many benefits
for using any scripting language in SSSD.
I'm careful about this as well. Yet, it is exciting :)
Moreover sssd is a security service. Such step could cause problems
with
security certifications.
Well, I know little of certification, and I'm worried about security too,
however, would C be safer, really?
Do you anticipate any specific security problems?
Thanks!
Nick
12:48 p.m.
On Tue, 2015-08-25 at 16:14 +0300, Nikolai Kondrashov wrote:
Hi Pavel,
On 08/24/2015 11:54 AM, Pavel Březina wrote:
> On 08/21/2015 07:01 PM, Nikolai Kondrashov wrote:
>> I might be in a strange and careless mood today, but here is something I
>> wanted to suggest since the time I saw the amount of logic that goes into
>> SSSD and is implemented in C.
>>
>> What if we implement some of the complicated logic inside SSSD in Lua [1]?
>
> can you tell us what features of Lua do you like and might help simplify
> SSSD? I don't know the language, just fast scrolled over the manual and seen
> few examples. It seems to be similar to javascript.
Most of all, I like its simplicity, then flexibility. And then how easy it is
to embed. The latter comes from simplicity, first of all.
I listed the features I think will help us in the other message in this
thread, but here they are again and then some:
* exception handling
(somewhat rudimentary, but workable)
This may be interesting, but we are pretty diligent I think, thanks to
the tevent_req design pattern, at surfacing error
* automatic memory management
(incremental mark-and-sweep with good controls, customizable allocator)
This is not really interesting as we already have talloc that does a
great job at keeping track of memory allocations and cleanups (esp if we
stop using the anti-pattern of allocating on NULL as discussed a few
times already).
* easier data massaging
(string operations, built-in (limited) regexes, etc.)
We do some limited string manipulation but not really much, what other
kind of data massaging does Lua provide that makes it worth ?
* brevity
(implicit dynamic typing, syntax sugar)
brevity is good for long-winded functions indeed.
* OOP support
(there are several libraries to choose from)
This may be good or bad, it all depends on what you need to do.
How does Lua work with asynchronous operations ? We have a ton of that
and unless Lua has native support it would be a huge mismatch.
I don't know enough JavaScript to really compare, but to me
knowing that it
has two pairs of equality operators [1] is quite off-putting.
I hope it doesn't compare much because javascript is awful, especially
for anything security oriented.
There is, however, a seemingly good comparison, which should give the
general
idea:
http://stackoverflow.com/a/1022683/1161045
To be honest a Javascript to Lua comparison doesn't really help much
understanding if Lua could be an asset for SSSD or not.
See above questions.
Simo.
Nick
[1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference#Equalit...
_______________________________________________
sssd-devel mailing list
sssd-devel(a)lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/sssd-devel
--
Simo Sorce * Red Hat, Inc * New York
1:26 p.m.
On 08/25/2015 08:48 PM, Simo Sorce wrote:
On Tue, 2015-08-25 at 16:14 +0300, Nikolai Kondrashov wrote:
> On 08/24/2015 11:54 AM, Pavel Březina wrote:
>> On 08/21/2015 07:01 PM, Nikolai Kondrashov wrote:
>>> I might be in a strange and careless mood today, but here is something I
>>> wanted to suggest since the time I saw the amount of logic that goes into
>>> SSSD and is implemented in C.
>>>
>>> What if we implement some of the complicated logic inside SSSD in Lua [1]?
>>
>> can you tell us what features of Lua do you like and might help simplify
>> SSSD? I don't know the language, just fast scrolled over the manual and seen
>> few examples. It seems to be similar to javascript.
>
> Most of all, I like its simplicity, then flexibility. And then how easy it is
> to embed. The latter comes from simplicity, first of all.
>
> I listed the features I think will help us in the other message in this
> thread, but here they are again and then some:
>
> * exception handling
> (somewhat rudimentary, but workable)
This may be interesting, but we are pretty diligent I think, thanks to
the tevent_req design pattern, at surfacing error
Main benefit of exceptions to me is not to have to propagate errors
explicitly.
> * automatic memory management
> (incremental mark-and-sweep with good controls, customizable allocator)
This is not really interesting as we already have talloc that does a
great job at keeping track of memory allocations and cleanups (esp if we
stop using the anti-pattern of allocating on NULL as discussed a few
times already).
You still need to do it explicitly and think about it. Not that much, of
course, and it's not that you don't have to think about it with a
mark-and-sweep GC at all. Still, in the majority of cases you don't have
to think about it, and you certainly don't have to assign memory to contexts
and keep them, nor pass them around.
> * easier data massaging
> (string operations, built-in (limited) regexes, etc.)
We do some limited string manipulation but not really much, what other
kind of data massaging does Lua provide that makes it worth ?
Well, can't think of many specifics, ATM, but let's say easier structure
creation for example. Just stuff data into tables in any place with a succinct
syntax. Especially useful for ad-hoc single-use structuring, where writing
constructor functions is not worth it, and allocating a structure and then
assigning fields in place is a pain.
> * brevity
> (implicit dynamic typing, syntax sugar)
brevity is good for long-winded functions indeed.
Well, it kind of works all through the code.
> * OOP support
> (there are several libraries to choose from)
This may be good or bad, it all depends on what you need to do.
Sure. Although having something is better than not having, theoretically.
Except diseases, of course :)
How does Lua work with asynchronous operations ? We have a ton of
that
and unless Lua has native support it would be a huge mismatch.
Now this is a good question. It depends on what you mean. If you mean central
loop with state machines underneath, then coroutines is an awesome solution.
If you mean OS threads, then there are several libraries that support it,
AFAIK. I haven't looked into them though.
We'll have to try it to really know. If you can give me an example, I can try
thinking about it.
> I don't know enough JavaScript to really compare, but to me
knowing that it
> has two pairs of equality operators [1] is quite off-putting.
I hope it doesn't compare much because javascript is awful, especially
for anything security oriented.
Now, this is interesting. Care to elaborate? Is there anything really in the
language itself and not the browser parts? Not that I like JavaScript myself,
just curious.
> There is, however, a seemingly good comparison, which should give
the general
> idea:
>
> http://stackoverflow.com/a/1022683/1161045
To be honest a Javascript to Lua comparison doesn't really help much
understanding if Lua could be an asset for SSSD or not.
Sure, I was answering the specific question from Pavel of how it compares to
JavaScript.
I think a lightweight embeddable language could be an asset to SSSD, in
general. It doesn't have to be Lua. I think Lua is a good candidate, but there
are other similar languages around, although probably not of such extended
vintage.
See above questions.
Thanks!
Nick
1:30 p.m.
On 08/25/2015 09:26 PM, Nikolai Kondrashov wrote:
I think a lightweight embeddable language could be an asset to SSSD,
in
general. It doesn't have to be Lua. I think Lua is a good candidate, but there
are other similar languages around, although probably not of such extended
vintage.
There is one question on StackOverflow about what to use instead of Lua, which
ends with a glowing feedback, I like it :)
http://stackoverflow.com/questions/4448835/alternatives-to-lua-as-an-embe...
Nick
2:19 p.m.
On Tue, 2015-08-25 at 21:26 +0300, Nikolai Kondrashov wrote:
On 08/25/2015 08:48 PM, Simo Sorce wrote:
> On Tue, 2015-08-25 at 16:14 +0300, Nikolai Kondrashov wrote:
>> On 08/24/2015 11:54 AM, Pavel Březina wrote:
>>> On 08/21/2015 07:01 PM, Nikolai Kondrashov wrote:
>>>> I might be in a strange and careless mood today, but here is something
I
>>>> wanted to suggest since the time I saw the amount of logic that goes
into
>>>> SSSD and is implemented in C.
>>>>
>>>> What if we implement some of the complicated logic inside SSSD in Lua
[1]?
>>>
>>> can you tell us what features of Lua do you like and might help simplify
>>> SSSD? I don't know the language, just fast scrolled over the manual and
seen
>>> few examples. It seems to be similar to javascript.
>>
>> Most of all, I like its simplicity, then flexibility. And then how easy it is
>> to embed. The latter comes from simplicity, first of all.
>>
>> I listed the features I think will help us in the other message in this
>> thread, but here they are again and then some:
>>
>> * exception handling
>> (somewhat rudimentary, but workable)
>
> This may be interesting, but we are pretty diligent I think, thanks to
> the tevent_req design pattern, at surfacing error
Main benefit of exceptions to me is not to have to propagate errors
explicitly.
>> * automatic memory management
>> (incremental mark-and-sweep with good controls, customizable allocator)
>
> This is not really interesting as we already have talloc that does a
> great job at keeping track of memory allocations and cleanups (esp if we
> stop using the anti-pattern of allocating on NULL as discussed a few
> times already).
You still need to do it explicitly and think about it. Not that much, of
course, and it's not that you don't have to think about it with a
mark-and-sweep GC at all.
I think a GC would be a regression to be honest,for the usage pattern of
SSSD a hierarchical allocator makes better use of memory.
Still, in the majority of cases you don't have
to think about it, and you certainly don't have to assign memory to contexts
and keep them, nor pass them around.
True, but then you do not ever think of memory allocation with usually
comical situations where you leak tremendous amounts of memory because
you keep some reference mistakenly. Once that happen finding out what is
happening and reviewing every single variable allocation to find out
what is going on is generally extremely hard. It's a trade off really.
>> * easier data massaging
>> (string operations, built-in (limited) regexes, etc.)
>
> We do some limited string manipulation but not really much, what other
> kind of data massaging does Lua provide that makes it worth ?
Well, can't think of many specifics, ATM, but let's say easier structure
creation for example. Just stuff data into tables in any place with a succinct
syntax. Especially useful for ad-hoc single-use structuring, where writing
constructor functions is not worth it, and allocating a structure and then
assigning fields in place is a pain.
Yes explicit memory management is a pain point in C, but in and on
itself does not justify changing sssd, so I'll discount this point under
the general "memory management is easier" argument.
>> * brevity
>> (implicit dynamic typing, syntax sugar)
>
> brevity is good for long-winded functions indeed.
Well, it kind of works all through the code.
Sometimes the brevity of scripting languages is obtained through
extremely inefficient code, with potentially unwanted side-effect (esp
wrt memory management). So it is not better in general, it is better for
the specific goal of making long functions more readable.
>> * OOP support
>> (there are several libraries to choose from)
>
> This may be good or bad, it all depends on what you need to do.
Sure. Although having something is better than not having, theoretically.
Except diseases, of course :)
As you see, sometimes having "something" is worse. For software that
works at lower levels OOP usually makes for software that is harder to
debug, so it may be an anti-feature.
> How does Lua work with asynchronous operations ? We have a ton
of that
> and unless Lua has native support it would be a huge mismatch.
Now this is a good question. It depends on what you mean. If you mean central
loop with state machines underneath, then coroutines is an awesome solution.
It is not clear to me that Lua's coroutines
(http://www.lua.org/pil/9.1.html) would do what asynchronous tevents do.
If you mean OS threads, then there are several libraries that support
it,
AFAIK. I haven't looked into them though.
No, I do not mean threads, in fact sssd is not thread safe. I mean I/O
based events where a socket file becoming readable makes the chain of
nested events keep going on until the innnermost completes it's work and
data is returned (potentially massaged all the way back) to the
outermost event. With control returning to the main loop every time an
I/O operation would block.
We'll have to try it to really know. If you can give me an
example, I can try
thinking about it.
Look at how tevent_req stuff works / see above.
>> I don't know enough JavaScript to really compare, but to
me knowing that it
>> has two pairs of equality operators [1] is quite off-putting.
>
> I hope it doesn't compare much because javascript is awful, especially
> for anything security oriented.
Now, this is interesting. Care to elaborate? Is there anything really in the
language itself and not the browser parts? Not that I like JavaScript myself,
just curious.
The browsers just implement the language, which is quite awful.
>> There is, however, a seemingly good comparison, which should
give the general
>> idea:
>>
>> http://stackoverflow.com/a/1022683/1161045
>
> To be honest a Javascript to Lua comparison doesn't really help much
> understanding if Lua could be an asset for SSSD or not.
Sure, I was answering the specific question from Pavel of how it compares to
JavaScript.
I think a lightweight embeddable language could be an asset to SSSD, in
general. It doesn't have to be Lua. I think Lua is a good candidate, but there
are other similar languages around, although probably not of such extended
vintage.
If we needed an embeddable language Lua would probably be a very good
candidate, it is not clear to me that the proposition adds enough value.
We'd really need to see an example albeit small of how, some task
currently done in sssd would translate in Lua, with which tradeoffs.
Simo.
--
Simo Sorce * Red Hat, Inc * New York
1:50 a.m.
On (25/08/15 19:45), Nikolai Kondrashov wrote:
On 08/25/2015 06:59 PM, Lukas Slebodnik wrote:
>On (25/08/15 15:57), Nikolai Kondrashov wrote:
>>Hi Simo,
>>
>>On 08/21/2015 09:37 PM, Simo Sorce wrote:
>>>On Fri, 2015-08-21 at 20:01 +0300, Nikolai Kondrashov wrote:
>>>>I might be in a strange and careless mood today, but here is something I
>>>>wanted to suggest since the time I saw the amount of logic that goes into
SSSD
>>>>and is implemented in C.
>>>>
>>>>What if we implement some of the complicated logic inside SSSD in Lua
[1]?
>>>
>>>can you provide an example of a piece of SSSD youd replace with Lua ?
>>>I am not asking for an implementation but a high level view of what a
>>>function looks like to day and what it would look like if we were going
>>>to use Lua (ideally with a list of primitives we'd still need to
>>>provide, to understand how much code Lua replaces for real.
>>
>>I don't have enough knowledge of SSSD to provide really meaningful examples.
>>I have to know developer pain points to really suggest an interesting
>>solution, and I have to really know the code to show good examples.
>>
>SSSD is written in asynchronous way and tightly coupled with tevent[1] and
>talloc. This is not use case for using scripting language. The ideal use
>case for scripting languages is to write higly optimized library in C +
>bindings. The scipting language would be used for replacing boilerplate code
>in C. However there are still few ways. You can write simple example of
>using lua scripts with tevent[1]. Then we can evaluate pros and cons of
>using Lua.
Now we're talking specifics, awesome! Alright, perhaps I'll do that. Care to
give a use case I should try to implement?
>The another way would be to replace tevent[1] with lua implementation of event loop.
>In another words, rewrite sssd from scratch.
Whoa, no way :)
>>Still, judging from the code distribution and overall architecture, I would
>>say the first candidates will be providers. They are very complicated and have
>>a relatively well defined interface to SSSD core. Most of them talk over
>>network and any slowdown from Lua will likely be swamped by network latency.
>>
>Providers are very complicated due to compicated servers (LDAP, IPA, AD)
>We will not be able to replace high level iplementation with scripting language
>in short time (Lua -> C) . In short time, there still will need to be two way
>comunication between C and Lua (Lua <-> C). Which would not be simplify code as
>the first way.
Sure. It's complicated. That's why a higher level language might be
beneficial. If we can cut off some parts of a provider cleanly (which is a
good idea for refactoring anyway), we can reimplement them in Lua with
relatively low overhead. Thus, going by pieces we can reduce the disruption.
We separated logic to small parts with tevent style.
* create request
* do staff
* register callback
* call callback after finishing asynchronous operation.
If we want to use tevent + lua there might not be a huge saving in LOC;
You safe few lines with lua code, but you add extra lines for bingins to
wrap tevent request. Maybe it would help to write some example.
BTW you might find simple example of tevent style in libtevent (samba) itself.
It might be faster way then extracting LDAP example from sssd.
>>I expect main savings to come from
>>exception support,
>scripting language might help
>
>>automatic memory management,
>talloc helps a lot even in C.
I agree. Still, it's explicit and needs taking care of, constantly.
Yes, but there is a huge simplification with hierachical allocator
+ registering destructors for complicated stuff.
So memory management is not a benefit.
>>simpler data massaging,
>What do you mean here?
The messy stuff. Parsing and gluing strings together, maintaining complicated
data structures, converting from one representation to another, etc.
Agree.
BTW gluing strings together is not very compicated with
talloc_asprintf; however parsing strings would be a much nicer.
>>and brevity of a scripting language (e.g. implicit, dynamic
typing).
>>
>This can be pros and cons in the same time.
Implicit, dynamic typing - yes. Brevity - not likely. You can always make it
more verbose, after all, but it is hard the other way around.
>Summary:
>This is an interesting idea; but currently I cannot see many benefits
>for using any scripting language in SSSD.
I'm careful about this as well. Yet, it is exciting :)
>Moreover sssd is a security service. Such step could cause problems with
>security certifications.
Well, I know little of certification, and I'm worried about security too,
however, would C be safer, really?
Do you anticipate any specific security problems?
I do not have anything specific;
It's just key factor which shoudl be considered for
such step as using scripting language in sssd.
You can write secure code even in C.
https://docs.fedoraproject.org/en-US/Fedora_Security_Team/1/html/Defensiv...
LS
4:28 a.m.
On 08/26/2015 09:50 AM, Lukas Slebodnik wrote:
On (25/08/15 19:45), Nikolai Kondrashov wrote:
> On 08/25/2015 06:59 PM, Lukas Slebodnik wrote:
>> On (25/08/15 15:57), Nikolai Kondrashov wrote:
>>> Still, judging from the code distribution and overall architecture, I would
>>> say the first candidates will be providers. They are very complicated and
have
>>> a relatively well defined interface to SSSD core. Most of them talk over
>>> network and any slowdown from Lua will likely be swamped by network latency.
>>>
>> Providers are very complicated due to compicated servers (LDAP, IPA, AD)
>> We will not be able to replace high level iplementation with scripting language
>> in short time (Lua -> C) . In short time, there still will need to be two way
>> comunication between C and Lua (Lua <-> C). Which would not be simplify
code as
>> the first way.
>
> Sure. It's complicated. That's why a higher level language might be
> beneficial. If we can cut off some parts of a provider cleanly (which is a
> good idea for refactoring anyway), we can reimplement them in Lua with
> relatively low overhead. Thus, going by pieces we can reduce the disruption.
>
We separated logic to small parts with tevent style.
* create request
* do staff
* register callback
* call callback after finishing asynchronous operation.
If we want to use tevent + lua there might not be a huge saving in LOC;
You safe few lines with lua code, but you add extra lines for bingins to
wrap tevent request. Maybe it would help to write some example.
BTW you might find simple example of tevent style in libtevent (samba) itself.
It might be faster way then extracting LDAP example from sssd.
Thanks a lot, Lukas. I'll see what can be done, once I have time.
I have some ideas already.
> Well, I know little of certification, and I'm worried about
security too,
> however, would C be safer, really?
>
> Do you anticipate any specific security problems?
I do not have anything specific; It's just key factor which shoudl be considered for
such step as using scripting language in sssd.
Alright, I'll do some research on this.
You can write secure code even in C.
https://docs.fedoraproject.org/en-US/Fedora_Security_Team/1/html/Defensiv...
Sure. The question is effort.
I should read that though.
Thanks!
Nick
8:10 a.m.
On 08/25/2015 10:19 PM, Simo Sorce wrote:
On Tue, 2015-08-25 at 21:26 +0300, Nikolai Kondrashov wrote:
> On 08/25/2015 08:48 PM, Simo Sorce wrote:
>> On Tue, 2015-08-25 at 16:14 +0300, Nikolai Kondrashov wrote:
>>> * automatic memory management
>>> (incremental mark-and-sweep with good controls, customizable
allocator)
>>
>> This is not really interesting as we already have talloc that does a
>> great job at keeping track of memory allocations and cleanups (esp if we
>> stop using the anti-pattern of allocating on NULL as discussed a few
>> times already).
>
> You still need to do it explicitly and think about it. Not that much, of
> course, and it's not that you don't have to think about it with a
> mark-and-sweep GC at all.
I think a GC would be a regression to be honest,for the usage pattern of
SSSD a hierarchical allocator makes better use of memory.
I agree, memory can be used more efficiently with a hierarchical allocator.
However, is saving memory a priority over lowering complexity for providers?
> Still, in the majority of cases you don't have
> to think about it, and you certainly don't have to assign memory to contexts
> and keep them, nor pass them around.
True, but then you do not ever think of memory allocation with usually
comical situations where you leak tremendous amounts of memory because
you keep some reference mistakenly. Once that happen finding out what is
happening and reviewing every single variable allocation to find out
what is going on is generally extremely hard. It's a trade off really.
Agree. Doesn't happen often, though. In fact, I can't remember it happening to
me. I had some difficulties bridging Lua and a C library (ClearSilver) memory
management, but it wasn't Lua's fault and it worked in the end.
>>> * brevity
>>> (implicit dynamic typing, syntax sugar)
>>
>> brevity is good for long-winded functions indeed.
>
> Well, it kind of works all through the code.
Sometimes the brevity of scripting languages is obtained through
extremely inefficient code, with potentially unwanted side-effect (esp
wrt memory management). So it is not better in general, it is better for
the specific goal of making long functions more readable.
Sure. We can only consider it better or worse in isolation from other factors.
>>> * OOP support
>>> (there are several libraries to choose from)
>>
>> This may be good or bad, it all depends on what you need to do.
>
> Sure. Although having something is better than not having, theoretically.
> Except diseases, of course :)
As you see, sometimes having "something" is worse. For software that
works at lower levels OOP usually makes for software that is harder to
debug, so it may be an anti-feature.
Sure. OOP might only be useful for higher-level logic, but I don't suggest we
use it otherwise. It was just a point that might help us.
>> How does Lua work with asynchronous operations ? We have a
ton of that
>> and unless Lua has native support it would be a huge mismatch.
>
> Now this is a good question. It depends on what you mean. If you mean central
> loop with state machines underneath, then coroutines is an awesome solution.
It is not clear to me that Lua's coroutines
(http://www.lua.org/pil/9.1.html) would do what asynchronous tevents do.
I'll try marrying them when I have time and we'll see.
> If you mean OS threads, then there are several libraries that
support it,
> AFAIK. I haven't looked into them though.
No, I do not mean threads, in fact sssd is not thread safe. I mean I/O
based events where a socket file becoming readable makes the chain of
nested events keep going on until the innnermost completes it's work and
data is returned (potentially massaged all the way back) to the
outermost event. With control returning to the main loop every time an
I/O operation would block.
Ah, good, I was worried we might need threads.
> We'll have to try it to really know. If you can give me an
example, I can try
> thinking about it.
Look at how tevent_req stuff works / see above.
I will, thank you.
>>> I don't know enough JavaScript to really compare, but
to me knowing that it
>>> has two pairs of equality operators [1] is quite off-putting.
>>
>> I hope it doesn't compare much because javascript is awful, especially
>> for anything security oriented.
>
> Now, this is interesting. Care to elaborate? Is there anything really in the
> language itself and not the browser parts? Not that I like JavaScript myself,
> just curious.
The browsers just implement the language, which is quite awful.
Well the fact that a browser would implement a scripting language is not that
bad, IMO, but the huge mess of the DOM and the APIs is what makes it very hard
to contain.
I would never suggest we put a browser, or DOM into sssd :)
>>> There is, however, a seemingly good comparison, which
should give the general
>>> idea:
>>>
>>> http://stackoverflow.com/a/1022683/1161045
>>
>> To be honest a Javascript to Lua comparison doesn't really help much
>> understanding if Lua could be an asset for SSSD or not.
>
> Sure, I was answering the specific question from Pavel of how it compares to
> JavaScript.
>
> I think a lightweight embeddable language could be an asset to SSSD, in
> general. It doesn't have to be Lua. I think Lua is a good candidate, but there
> are other similar languages around, although probably not of such extended
> vintage.
If we needed an embeddable language Lua would probably be a very good
candidate, it is not clear to me that the proposition adds enough value.
We'd really need to see an example albeit small of how, some task
currently done in sssd would translate in Lua, with which tradeoffs.
Alright, I'll try to provide one, if even just for the fun of it, but it
won't be soon.
Thank you!
Nick
9:23 a.m.
On Wed, 2015-08-26 at 16:10 +0300, Nikolai Kondrashov wrote:
On 08/25/2015 10:19 PM, Simo Sorce wrote:
> On Tue, 2015-08-25 at 21:26 +0300, Nikolai Kondrashov wrote:
>> On 08/25/2015 08:48 PM, Simo Sorce wrote:
>>> On Tue, 2015-08-25 at 16:14 +0300, Nikolai Kondrashov wrote:
>>>> * automatic memory management
>>>> (incremental mark-and-sweep with good controls, customizable
allocator)
>>>
>>> This is not really interesting as we already have talloc that does a
>>> great job at keeping track of memory allocations and cleanups (esp if we
>>> stop using the anti-pattern of allocating on NULL as discussed a few
>>> times already).
>>
>> You still need to do it explicitly and think about it. Not that much, of
>> course, and it's not that you don't have to think about it with a
>> mark-and-sweep GC at all.
>
> I think a GC would be a regression to be honest,for the usage pattern of
> SSSD a hierarchical allocator makes better use of memory.
I agree, memory can be used more efficiently with a hierarchical allocator.
However, is saving memory a priority over lowering complexity for providers?
It is important, yes, because this is a system service that always runs
on the machine. We cannot explode the memory requirements or it will
negatively affect the whole system, we are already borderline for some
use cases.
>> Still, in the majority of cases you don't have
>> to think about it, and you certainly don't have to assign memory to
contexts
>> and keep them, nor pass them around.
>
> True, but then you do not ever think of memory allocation with usually
> comical situations where you leak tremendous amounts of memory because
> you keep some reference mistakenly. Once that happen finding out what is
> happening and reviewing every single variable allocation to find out
> what is going on is generally extremely hard. It's a trade off really.
Agree. Doesn't happen often, though. In fact, I can't remember it happening to
me. I had some difficulties bridging Lua and a C library (ClearSilver) memory
management, but it wasn't Lua's fault and it worked in the end.
>>>> * brevity
>>>> (implicit dynamic typing, syntax sugar)
>>>
>>> brevity is good for long-winded functions indeed.
>>
>> Well, it kind of works all through the code.
>
> Sometimes the brevity of scripting languages is obtained through
> extremely inefficient code, with potentially unwanted side-effect (esp
> wrt memory management). So it is not better in general, it is better for
> the specific goal of making long functions more readable.
Sure. We can only consider it better or worse in isolation from other factors.
>>>> * OOP support
>>>> (there are several libraries to choose from)
>>>
>>> This may be good or bad, it all depends on what you need to do.
>>
>> Sure. Although having something is better than not having, theoretically.
>> Except diseases, of course :)
>
> As you see, sometimes having "something" is worse. For software that
> works at lower levels OOP usually makes for software that is harder to
> debug, so it may be an anti-feature.
Sure. OOP might only be useful for higher-level logic, but I don't suggest we
use it otherwise. It was just a point that might help us.
>>> How does Lua work with asynchronous operations ? We have a ton of that
>>> and unless Lua has native support it would be a huge mismatch.
>>
>> Now this is a good question. It depends on what you mean. If you mean central
>> loop with state machines underneath, then coroutines is an awesome solution.
>
> It is not clear to me that Lua's coroutines
> (http://www.lua.org/pil/9.1.html) would do what asynchronous tevents do.
I'll try marrying them when I have time and we'll see.
>> If you mean OS threads, then there are several libraries that support it,
>> AFAIK. I haven't looked into them though.
>
> No, I do not mean threads, in fact sssd is not thread safe. I mean I/O
> based events where a socket file becoming readable makes the chain of
> nested events keep going on until the innnermost completes it's work and
> data is returned (potentially massaged all the way back) to the
> outermost event. With control returning to the main loop every time an
> I/O operation would block.
Ah, good, I was worried we might need threads.
>> We'll have to try it to really know. If you can give me an example, I can
try
>> thinking about it.
>
> Look at how tevent_req stuff works / see above.
I will, thank you.
>>>> I don't know enough JavaScript to really compare, but to me knowing
that it
>>>> has two pairs of equality operators [1] is quite off-putting.
>>>
>>> I hope it doesn't compare much because javascript is awful, especially
>>> for anything security oriented.
>>
>> Now, this is interesting. Care to elaborate? Is there anything really in the
>> language itself and not the browser parts? Not that I like JavaScript myself,
>> just curious.
>
> The browsers just implement the language, which is quite awful.
Well the fact that a browser would implement a scripting language is not that
bad, IMO, but the huge mess of the DOM and the APIs is what makes it very hard
to contain.
I would never suggest we put a browser, or DOM into sssd :)
>>>> There is, however, a seemingly good comparison, which should give the
general
>>>> idea:
>>>>
>>>> http://stackoverflow.com/a/1022683/1161045
>>>
>>> To be honest a Javascript to Lua comparison doesn't really help much
>>> understanding if Lua could be an asset for SSSD or not.
>>
>> Sure, I was answering the specific question from Pavel of how it compares to
>> JavaScript.
>>
>> I think a lightweight embeddable language could be an asset to SSSD, in
>> general. It doesn't have to be Lua. I think Lua is a good candidate, but
there
>> are other similar languages around, although probably not of such extended
>> vintage.
>
> If we needed an embeddable language Lua would probably be a very good
> candidate, it is not clear to me that the proposition adds enough value.
>
> We'd really need to see an example albeit small of how, some task
> currently done in sssd would translate in Lua, with which tradeoffs.
Alright, I'll try to provide one, if even just for the fun of it, but it
won't be soon.
Sure, np, thanks.
Simo.
--
Simo Sorce * Red Hat, Inc * New York
3165
days inactive
3170
days old
sssd-devel@lists.fedorahosted.org
23 comments
5 participants
participants (5)
-
Lukas Slebodnik -
Michal Židek -
Nikolai Kondrashov -
Pavel Březina -
Simo Sorce