Hi all,
Background: https://pagure.io/389-ds-base/issue/51140
Ludwig and Mark both raised some good points here. First is that features today (EntryUUID, LDAPSsoToken) are both locked behind another language. Rightly so, they shouldn't be hidden here forever. As well, Mark wants to ask what it would take to enable by default.
** Why did I use the fedse.c to load a plugin entry?
Because we don't (fully) support startup migrations yet. Mark has privately reminded me that he has reviewed https://pagure.io/389-ds-base/pull-request/49579 and that I should revive it. This v4 plugin adds a stateful assertion of entries allowing better migrations beyond what fedse.c's bootstrap can achieve. IE we can create an entry if it does not exist, and modify it partially if an admin has changed it (ie we can tweak defaults but without affecing say pluginEnabled). There was an issue viktor raised about pbkdf2 where it could not be disabled, and this would resolve it. Most likely, I will break the on startup migrations out from the v4 plugin patch set, and make it it's own change, and move some of the content from fedse.c and friends into this.
** TODO list to get rust enabled by default.
I think there is still a bit of work to do to enable by default, but I think we are pretty close. Roughly ordered, this is the list of things that has to happen to let us enable rust by default.
- everyone -> test that you can build and run tests with --enable-rust --disable-asan in your development environment so that we can work out and issues that may exist for us as developers. - william -> fix the intentional name leak in the rust slapi plugin interface to use lazy_static. Today this triggers LSAN which breaks basic test suites. - william -> clean up libsds linking and some of the related elements - william -> revive and make on upgrade configs better (see above), potentially break it out from the v4 plugin patch. - william + mark -> check that our respective major distros/releases can build with --enable-rust in release rpms - william + viktor -> check that we can pass with --enable-rust and --enable-asan, especially in CI - anyone interested -> update wiki contributing guide to include rust as a requirement - (optional) anyone interested (but not william) -> add a section to the wiki on using rust in development - team -> agree we are happy, and make configure.ac have --enable-rust by default - team -> after a few weeks / months, remove the ifdefs, duplicate C versions of Rust features, and enable/disable features from configure.ac
Does this seem reasonable to everyone? I really want to make sure we do this right as a team, and we are all happy with it (I don't want a repeat of nunc-stans ;) ). If we agree on this, I will open some tickets up, add the relevant blocks/ordering and assign out the work.
Thank you all!
— Sincerely,
William Brown
Senior Software Engineer, 389 Directory Server SUSE Labs
On 6/11/20 8:15 PM, William Brown wrote:
Hi all,
Background: https://pagure.io/389-ds-base/issue/51140
Ludwig and Mark both raised some good points here. First is that features today (EntryUUID, LDAPSsoToken) are both locked behind another language. Rightly so, they shouldn't be hidden here forever. As well, Mark wants to ask what it would take to enable by default.
** Why did I use the fedse.c to load a plugin entry?
Because we don't (fully) support startup migrations yet. Mark has privately reminded me that he has reviewed https://pagure.io/389-ds-base/pull-request/49579 and that I should revive it. This v4 plugin adds a stateful assertion of entries allowing better migrations beyond what fedse.c's bootstrap can achieve. IE we can create an entry if it does not exist, and modify it partially if an admin has changed it (ie we can tweak defaults but without affecing say pluginEnabled). There was an issue viktor raised about pbkdf2 where it could not be disabled, and this would resolve it. Most likely, I will break the on startup migrations out from the v4 plugin patch set, and make it it's own change, and move some of the content from fedse.c and friends into this.
** TODO list to get rust enabled by default.
I think there is still a bit of work to do to enable by default, but I think we are pretty close. Roughly ordered, this is the list of things that has to happen to let us enable rust by default.
- everyone -> test that you can build and run tests with --enable-rust --disable-asan in your development environment so that we can work out and issues that may exist for us as developers.
I meant to try this again to confirm, but it worked for me a few weeks ago on Fedora. The build took a lot longer :-( but it worked ;-)
- william -> fix the intentional name leak in the rust slapi plugin interface to use lazy_static. Today this triggers LSAN which breaks basic test suites.
- william -> clean up libsds linking and some of the related elements
- william -> revive and make on upgrade configs better (see above), potentially break it out from the v4 plugin patch.
- william + mark -> check that our respective major distros/releases can build with --enable-rust in release rpms
As long as "make -f rpm.mk dist-bz2" does all the cargo building it should work fine on Fedora and RHEL.
- william + viktor -> check that we can pass with --enable-rust and --enable-asan, especially in CI
- anyone interested -> update wiki contributing guide to include rust as a requirement
- (optional) anyone interested (but not william) -> add a section to the wiki on using rust in development
- team -> agree we are happy, and make configure.ac have --enable-rust by default
- team -> after a few weeks / months, remove the ifdefs, duplicate C versions of Rust features, and enable/disable features from configure.ac
Does this seem reasonable to everyone? I really want to make sure we do this right as a team, and we are all happy with it (I don't want a repeat of nunc-stans ;) ). If we agree on this, I will open some tickets up, add the relevant blocks/ordering and assign out the work.
Thanks for getting this discussion started! I think we already have most of this in place to use rust by default today, but we need to do some more testing, etc.
Thanks,
Mark
Thank you all!
— Sincerely,
William Brown
Senior Software Engineer, 389 Directory Server SUSE Labs _______________________________________________ 389-devel mailing list -- 389-devel@lists.fedoraproject.org To unsubscribe send an email to 389-devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject....
On 16 Jun 2020, at 09:37, Mark Reynolds mreynolds@redhat.com wrote:
On 6/11/20 8:15 PM, William Brown wrote:
Hi all,
Background: https://pagure.io/389-ds-base/issue/51140
Ludwig and Mark both raised some good points here. First is that features today (EntryUUID, LDAPSsoToken) are both locked behind another language. Rightly so, they shouldn't be hidden here forever. As well, Mark wants to ask what it would take to enable by default.
** Why did I use the fedse.c to load a plugin entry?
Because we don't (fully) support startup migrations yet. Mark has privately reminded me that he has reviewed https://pagure.io/389-ds-base/pull-request/49579 and that I should revive it. This v4 plugin adds a stateful assertion of entries allowing better migrations beyond what fedse.c's bootstrap can achieve. IE we can create an entry if it does not exist, and modify it partially if an admin has changed it (ie we can tweak defaults but without affecing say pluginEnabled). There was an issue viktor raised about pbkdf2 where it could not be disabled, and this would resolve it. Most likely, I will break the on startup migrations out from the v4 plugin patch set, and make it it's own change, and move some of the content from fedse.c and friends into this.
** TODO list to get rust enabled by default.
I think there is still a bit of work to do to enable by default, but I think we are pretty close. Roughly ordered, this is the list of things that has to happen to let us enable rust by default.
- everyone -> test that you can build and run tests with --enable-rust --disable-asan in your development environment so that we can work out and issues that may exist for us as developers.
I meant to try this again to confirm, but it worked for me a few weeks ago on Fedora. The build took a lot longer :-( but it worked ;-)
It'll get quicker with some of the cleanups I want to do, but release builds with rust will be a lot slower due to the fact they can perform more aggressive optimisation because of the stronger type rules AND they can also do stuff like LTO where we don't in 389 atm.
https://gcc.gnu.org/wiki/LinkTimeOptimization
- william -> fix the intentional name leak in the rust slapi plugin interface to use lazy_static. Today this triggers LSAN which breaks basic test suites.
- william -> clean up libsds linking and some of the related elements
- william -> revive and make on upgrade configs better (see above), potentially break it out from the v4 plugin patch.
- william + mark -> check that our respective major distros/releases can build with --enable-rust in release rpms
As long as "make -f rpm.mk dist-bz2" does all the cargo building it should work fine on Fedora and RHEL.
Sure, we'll need to test that at some point to be sure. I know it already works in SUSE :)
- william + viktor -> check that we can pass with --enable-rust and --enable-asan, especially in CI
- anyone interested -> update wiki contributing guide to include rust as a requirement
- (optional) anyone interested (but not william) -> add a section to the wiki on using rust in development
- team -> agree we are happy, and make configure.ac have --enable-rust by default
- team -> after a few weeks / months, remove the ifdefs, duplicate C versions of Rust features, and enable/disable features from configure.ac
Does this seem reasonable to everyone? I really want to make sure we do this right as a team, and we are all happy with it (I don't want a repeat of nunc-stans ;) ). If we agree on this, I will open some tickets up, add the relevant blocks/ordering and assign out the work.
Thanks for getting this discussion started! I think we already have most of this in place to use rust by default today, but we need to do some more testing, etc.
For sure!
Also worth noting that don't have to be the one to do all the "william" items, if other people want to step up they can. I need to tidy up this syncrepl stuff and finish up openldap migration stuff, but I think the rust tools will be a valuable part of that openldap migration so I will do it sooner than later :)
Thanks,
Mark
Thank you all!
— Sincerely,
William Brown
Senior Software Engineer, 389 Directory Server SUSE Labs _______________________________________________ 389-devel mailing list -- 389-devel@lists.fedoraproject.org To unsubscribe send an email to 389-devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject....
--
389 Directory Server Development Team
— Sincerely,
William Brown
Senior Software Engineer, 389 Directory Server SUSE Labs
Hi everyone,
I'm once again here to bring up my favourite topic, of rust-in-ds. Slow and steady progress has been made, and it would be good to update the situation here.
Completed Items are:
- william -> fix the intentional name leak in the rust slapi plugin interface to use lazy_static. Today this triggers LSAN which breaks basic test suites.
- william -> revive and make on upgrade configs better (see above), potentially break it out from the v4 plugin patch.
Todo Soon:
- everyone -> test that you can build and run tests with --enable-rust --disable-asan in your development environment so that we can work out and issues that may exist for us as developers.
I meant to try this again to confirm, but it worked for me a few weeks ago on Fedora. The build took a lot longer :-( but it worked ;-)
- william -> clean up libsds linking and some of the related elements
- william + mark -> check that our respective major distros/releases can build with --enable-rust in release rpms
As long as "make -f rpm.mk dist-bz2" does all the cargo building it should work fine on Fedora and RHEL.
Todo a bit later
- william + viktor -> check that we can pass with --enable-rust and --enable-asan, especially in CI
- anyone interested -> update wiki contributing guide to include rust as a requirement
- (optional) anyone interested (but not william) -> add a section to the wiki on using rust in development
- team -> agree we are happy, and make configure.ac have --enable-rust by default
- team -> after a few weeks / months, remove the ifdefs, duplicate C versions of Rust features, and enable/disable features from configure.ac
I'm hoping that if we start to look at some of these items sooner, we could think about this for 1.4.4 (?).
At the moment the major item for all of us would be to test that we can build in our development environments with rust, as that's a pretty major thing we need to pass for us to continue :)
Thanks!
— Sincerely,
William Brown
Senior Software Engineer, 389 Directory Server SUSE Labs, Australia
389-devel@lists.fedoraproject.org