Hi,
as we're stabilizing the 1.13 branch and before we plan what we want to work on during the 1.14 development, we should use that time to write some more tests!
Here are some areas where we could add tests. Please discuss or add your ideas, I would like to turn this list into tickets we can start implementing:
* Extend the LDAP provider tests with more dynamic test cases. - add a user to a group, run sss_cache, assert id user displays the new group and getent group displays the new member - conversely with removing users from groups * Background refresh - could be built atop the LDAP NSS tests as well. I think we have all the infrastructure in place. * Local overrides integration test: - this could be relatively easy, just call the overrides tool and request the entry. Could be built atop the existing LDAP tests or even use the local provider. * Add a KDC - until we have a pam_wrapper, this would only be useful to test ldap_child, but adding the KDC instantiation might be worth it nonetheless - there is a protorype of KDC instantiation on the list for some time now, since we enabled rootless SSSD * IFP - could we reuse the existing sbus tests to spawn a custom bus? * SUDO - can we trick sudo into connecting to our test sssd instance?
I think the order of priority is roughly as above. I think the LDAP provider is critical enough to be well tested. The refresh and local override tests might be nice to have because we would be refactoring the NSS responder in 1.14, so we should have it tested.
I'll be happy to hear other opionions, though!
On 08/20/2015 12:38 AM, Jakub Hrozek wrote:
as we're stabilizing the 1.13 branch and before we plan what we want to work on during the 1.14 development, we should use that time to write some more tests!
Here are some areas where we could add tests. Please discuss or add your ideas, I would like to turn this list into tickets we can start implementing:
- Extend the LDAP provider tests with more dynamic test cases.
- add a user to a group, run sss_cache, assert id user displays the new group and getent group displays the new member
- conversely with removing users from groups
There are some more basic tests which can be implemented, on my original design page: https://fedorahosted.org/sssd/wiki/DesignDocs/CwrapLDAP
I can help with this.
- Background refresh
- could be built atop the LDAP NSS tests as well. I think we have all the infrastructure in place.
Caching/refreshing tests would be a very useful thing, I agree. We'd have to be careful with timing, though, so they're robust enough under load on our CI setup.
- Local overrides integration test:
- this could be relatively easy, just call the overrides tool and request the entry. Could be built atop the existing LDAP tests or even use the local provider.
I can help with this.
- Add a KDC
- until we have a pam_wrapper, this would only be useful to test ldap_child, but adding the KDC instantiation might be worth it nonetheless
- there is a protorype of KDC instantiation on the list for some time now, since we enabled rootless SSSD
I can try helping with this.
- IFP - could we reuse the existing sbus tests to spawn a custom bus?
- SUDO - can we trick sudo into connecting to our test sssd instance?
This is interesting. Wouldn't we have to have some sort of nss_wrapper here too, as we can't write to /etc/nsswitch.conf? Would sudo be fine with uid_wrapper? I wrote a ton of sudo tests in QE, so I can at least help with the test plan.
Overall, I think I will be able to spare about two weeks after our autumn get-together to help with writing integration tests, maybe more later. Otherwise I'll need to work on session recording before that, so I have something to demo and talk about.
I think the order of priority is roughly as above. I think the LDAP provider is critical enough to be well tested. The refresh and local override tests might be nice to have because we would be refactoring the NSS responder in 1.14, so we should have it tested.
I'll be happy to hear other opionions, though!
Thanks for bringing this up Jakub! Personally, I'm always glad to see more integration tests upstream.
Nick
On Fri, Aug 21, 2015 at 04:24:13PM +0300, Nikolai Kondrashov wrote:
On 08/20/2015 12:38 AM, Jakub Hrozek wrote:
as we're stabilizing the 1.13 branch and before we plan what we want to work on during the 1.14 development, we should use that time to write some more tests!
Here are some areas where we could add tests. Please discuss or add your ideas, I would like to turn this list into tickets we can start implementing:
- Extend the LDAP provider tests with more dynamic test cases.
- add a user to a group, run sss_cache, assert id user displays the new group and getent group displays the new member
- conversely with removing users from groups
There are some more basic tests which can be implemented, on my original design page: https://fedorahosted.org/sssd/wiki/DesignDocs/CwrapLDAP
I can help with this.
Thank you very much for the help offer. I know you're also involved in some other projects -- do you think you'll have time for these in the near future? No is a perfectly acceptable answer, I'm mostly trying to find out the priorities and split the work..
- Background refresh
- could be built atop the LDAP NSS tests as well. I think we have all the infrastructure in place.
Caching/refreshing tests would be a very useful thing, I agree. We'd have to be careful with timing, though, so they're robust enough under load on our CI setup.
- Local overrides integration test:
- this could be relatively easy, just call the overrides tool and request the entry. Could be built atop the existing LDAP tests or even use the local provider.
I can help with this.
- Add a KDC
- until we have a pam_wrapper, this would only be useful to test ldap_child, but adding the KDC instantiation might be worth it nonetheless
- there is a protorype of KDC instantiation on the list for some time now, since we enabled rootless SSSD
I can try helping with this.
- IFP - could we reuse the existing sbus tests to spawn a custom bus?
- SUDO - can we trick sudo into connecting to our test sssd instance?
This is interesting. Wouldn't we have to have some sort of nss_wrapper here too, as we can't write to /etc/nsswitch.conf? Would sudo be fine with uid_wrapper? I wrote a ton of sudo tests in QE, so I can at least help with the test plan.
Overall, I think I will be able to spare about two weeks after our autumn get-together to help with writing integration tests, maybe more later. Otherwise I'll need to work on session recording before that, so I have something to demo and talk about.
I think the order of priority is roughly as above. I think the LDAP provider is critical enough to be well tested. The refresh and local override tests might be nice to have because we would be refactoring the NSS responder in 1.14, so we should have it tested.
I'll be happy to hear other opionions, though!
Thanks for bringing this up Jakub! Personally, I'm always glad to see more integration tests upstream.
Nick
On 08/31/2015 01:32 PM, Jakub Hrozek wrote:
On Fri, Aug 21, 2015 at 04:24:13PM +0300, Nikolai Kondrashov wrote:
On 08/20/2015 12:38 AM, Jakub Hrozek wrote:
as we're stabilizing the 1.13 branch and before we plan what we want to work on during the 1.14 development, we should use that time to write some more tests!
Here are some areas where we could add tests. Please discuss or add your ideas, I would like to turn this list into tickets we can start implementing:
- Extend the LDAP provider tests with more dynamic test cases.
- add a user to a group, run sss_cache, assert id user displays the new group and getent group displays the new member
- conversely with removing users from groups
There are some more basic tests which can be implemented, on my original design page: https://fedorahosted.org/sssd/wiki/DesignDocs/CwrapLDAP
I can help with this.
Thank you very much for the help offer. I know you're also involved in some other projects -- do you think you'll have time for these in the near future? No is a perfectly acceptable answer, I'm mostly trying to find out the priorities and split the work..
Sure, I'll likely be able to spare about two weeks after our meetings next week.
Nick
On Mon, Aug 31, 2015 at 01:46:11PM +0300, Nikolai Kondrashov wrote:
On 08/31/2015 01:32 PM, Jakub Hrozek wrote:
On Fri, Aug 21, 2015 at 04:24:13PM +0300, Nikolai Kondrashov wrote:
On 08/20/2015 12:38 AM, Jakub Hrozek wrote:
as we're stabilizing the 1.13 branch and before we plan what we want to work on during the 1.14 development, we should use that time to write some more tests!
Here are some areas where we could add tests. Please discuss or add your ideas, I would like to turn this list into tickets we can start implementing:
- Extend the LDAP provider tests with more dynamic test cases.
- add a user to a group, run sss_cache, assert id user displays the new group and getent group displays the new member
- conversely with removing users from groups
There are some more basic tests which can be implemented, on my original design page: https://fedorahosted.org/sssd/wiki/DesignDocs/CwrapLDAP
I can help with this.
Thank you very much for the help offer. I know you're also involved in some other projects -- do you think you'll have time for these in the near future? No is a perfectly acceptable answer, I'm mostly trying to find out the priorities and split the work..
Sure, I'll likely be able to spare about two weeks after our meetings next week.
Awesome, thank you very much. Then I would suggest you help with the LDAP part as you're familiar with the code already. Would that work for you? Or do you think it would be better the other way around so that more developers are aware of the LDAP tests?
On 08/31/2015 02:19 PM, Jakub Hrozek wrote:
On Mon, Aug 31, 2015 at 01:46:11PM +0300, Nikolai Kondrashov wrote:
On 08/31/2015 01:32 PM, Jakub Hrozek wrote:
On Fri, Aug 21, 2015 at 04:24:13PM +0300, Nikolai Kondrashov wrote:
On 08/20/2015 12:38 AM, Jakub Hrozek wrote:
as we're stabilizing the 1.13 branch and before we plan what we want to work on during the 1.14 development, we should use that time to write some more tests!
Here are some areas where we could add tests. Please discuss or add your ideas, I would like to turn this list into tickets we can start implementing:
- Extend the LDAP provider tests with more dynamic test cases.
- add a user to a group, run sss_cache, assert id user displays the new group and getent group displays the new member
- conversely with removing users from groups
There are some more basic tests which can be implemented, on my original design page: https://fedorahosted.org/sssd/wiki/DesignDocs/CwrapLDAP
I can help with this.
Thank you very much for the help offer. I know you're also involved in some other projects -- do you think you'll have time for these in the near future? No is a perfectly acceptable answer, I'm mostly trying to find out the priorities and split the work..
Sure, I'll likely be able to spare about two weeks after our meetings next week.
Awesome, thank you very much. Then I would suggest you help with the LDAP part as you're familiar with the code already. Would that work for you? Or do you think it would be better the other way around so that more developers are aware of the LDAP tests?
Sure, I'm fine with doing the LDAP tests.
Nick
On (19/08/15 23:38), Jakub Hrozek wrote:
Hi,
as we're stabilizing the 1.13 branch and before we plan what we want to work on during the 1.14 development, we should use that time to write some more tests!
Here are some areas where we could add tests. Please discuss or add your ideas, I would like to turn this list into tickets we can start implementing:
- Extend the LDAP provider tests with more dynamic test cases.
- add a user to a group, run sss_cache, assert id user displays the new group and getent group displays the new member
- conversely with removing users from groups
- Background refresh
- could be built atop the LDAP NSS tests as well. I think we have all the infrastructure in place.
- Local overrides integration test:
- this could be relatively easy, just call the overrides tool and request the entry. Could be built atop the existing LDAP tests or even use the local provider.
Firstly, we need to prepare test cases for this feature. We can inspire in downstream ipa-views test case. After review of test cases we can split it to several tickets.
- Add a KDC
- until we have a pam_wrapper, this would only be useful to test ldap_child, but adding the KDC instantiation might be worth it nonetheless
- there is a protorype of KDC instantiation on the list for some time now, since we enabled rootless SSSD
- IFP - could we reuse the existing sbus tests to spawn a custom bus?
Someone need to look into running dbus-daemon in cwrap env.
Then we need prepare test cases ...
- SUDO - can we trick sudo into connecting to our test sssd instance?
I think the order of priority is roughly as above. I think the LDAP provider is critical enough to be well tested. The refresh and local override tests might be nice to have because we would be refactoring the NSS responder in 1.14, so we should have it tested.
I'll be happy to hear other opionions, though!
We can try to prepare fake AD in openldap (if possible) and test basic functionality with ldap provider.
But we need to file tickets for each idea. (+ maybe assign someone)
LS
On Fri, Aug 28, 2015 at 03:16:57PM +0200, Lukas Slebodnik wrote:
On (19/08/15 23:38), Jakub Hrozek wrote:
Hi,
as we're stabilizing the 1.13 branch and before we plan what we want to work on during the 1.14 development, we should use that time to write some more tests!
Here are some areas where we could add tests. Please discuss or add your ideas, I would like to turn this list into tickets we can start implementing:
- Extend the LDAP provider tests with more dynamic test cases.
- add a user to a group, run sss_cache, assert id user displays the new group and getent group displays the new member
- conversely with removing users from groups
- Background refresh
- could be built atop the LDAP NSS tests as well. I think we have all the infrastructure in place.
- Local overrides integration test:
- this could be relatively easy, just call the overrides tool and request the entry. Could be built atop the existing LDAP tests or even use the local provider.
Firstly, we need to prepare test cases for this feature. We can inspire in downstream ipa-views test case. After review of test cases we can split it to several tickets.
We already do: https://fedorahosted.org/sssd/ticket/2732
It 'just' needs an assignee..
- Add a KDC
- until we have a pam_wrapper, this would only be useful to test ldap_child, but adding the KDC instantiation might be worth it nonetheless
- there is a protorype of KDC instantiation on the list for some time now, since we enabled rootless SSSD
- IFP - could we reuse the existing sbus tests to spawn a custom bus?
Someone need to look into running dbus-daemon in cwrap env.
Then we need prepare test cases ...
Well, the first testcase could be to run introspection -- that would validate the bus is accessible.
It could be the first ticket, if you agree, I can file a ticket and assign it?
- SUDO - can we trick sudo into connecting to our test sssd instance?
I think the order of priority is roughly as above. I think the LDAP provider is critical enough to be well tested. The refresh and local override tests might be nice to have because we would be refactoring the NSS responder in 1.14, so we should have it tested.
I'll be happy to hear other opionions, though!
We can try to prepare fake AD in openldap (if possible) and test basic functionality with ldap provider.
I would like to talk to Andreas next week and ask about Samba. I hope this would be much easier and more robust.
But we need to file tickets for each idea. (+ maybe assign someone)
Yes, this e-mail was about
On Mon, Aug 31, 2015 at 12:31:03PM +0200, Jakub Hrozek wrote:
Yes, this e-mail was about
Sorry, I fat-fingered the first mail. This e-mail was about where should we concentrate our effort to add more tests in the next couple of weeks as we stabilize 1.13 and before we start on 1.14.
On (31/08/15 12:31), Jakub Hrozek wrote:
On Fri, Aug 28, 2015 at 03:16:57PM +0200, Lukas Slebodnik wrote:
On (19/08/15 23:38), Jakub Hrozek wrote:
Hi,
as we're stabilizing the 1.13 branch and before we plan what we want to work on during the 1.14 development, we should use that time to write some more tests!
Here are some areas where we could add tests. Please discuss or add your ideas, I would like to turn this list into tickets we can start implementing:
- Extend the LDAP provider tests with more dynamic test cases.
- add a user to a group, run sss_cache, assert id user displays the new group and getent group displays the new member
- conversely with removing users from groups
There are some tests on list, but still it wuld be good to se a test plan.
- Background refresh
- could be built atop the LDAP NSS tests as well. I think we have all the infrastructure in place.
- Local overrides integration test:
- this could be relatively easy, just call the overrides tool and request the entry. Could be built atop the existing LDAP tests or even use the local provider.
Firstly, we need to prepare test cases for this feature. We can inspire in downstream ipa-views test case. After review of test cases we can split it to several tickets.
We already do: https://fedorahosted.org/sssd/ticket/2732
It 'just' needs an assignee..
IMHo this should be a very low priority ticket. It's already tested by downstream and they will have automated tests. We should focus to other parts which cannot be easily tested (whithout greping log files or touching/checking internals in sysdb) https://fedorahosted.org/sssd/ticket/2697
BTW if the tests were ready together with feature it would be something different. It would not be a duplicated effort.
If we really want to duplicate effort then it would be better to see/review a test plan for local views. Transforming to pytest should be a trivial task. Should I file a ticket?
- Add a KDC
- until we have a pam_wrapper, this would only be useful to test ldap_child, but adding the KDC instantiation might be worth it nonetheless
- there is a protorype of KDC instantiation on the list for some time now, since we enabled rootless SSSD
https://fedorahosted.org/sssd/ticket/2822
- IFP - could we reuse the existing sbus tests to spawn a custom bus?
Someone need to look into running dbus-daemon in cwrap env.
Then we need prepare test cases ...
Well, the first testcase could be to run introspection -- that would validate the bus is accessible.
It could be the first ticket, if you agree, I can file a ticket and assign it?
cwrap test for IFP should have the highest priority because it's not tested by downstream.
https://fedorahosted.org/sssd/ticket/2823 https://fedorahosted.org/sssd/ticket/2824 https://fedorahosted.org/sssd/ticket/2825
The first two tickets can be done in parallel.
- SUDO - can we trick sudo into connecting to our test sssd instance?
I thought it could be tested using pam_wrapper I filed a generat ticket for authentication + authorisation https://fedorahosted.org/sssd/ticket/2821
It can be split into testing pam provider and sudo provider.
I think the order of priority is roughly as above. I think the LDAP provider is critical enough to be well tested. The refresh and local override tests might be nice to have because we would be refactoring the NSS responder in 1.14, so we should have it tested.
I'll be happy to hear other opionions, though!
We can try to prepare fake AD in openldap (if possible) and test basic functionality with ldap provider.
I would like to talk to Andreas next week and ask about Samba. I hope this would be much easier and more robust.
Just for the record, here is a link to samba4-dc tracking ticket. https://fedorahosted.org/sssd/ticket/2818
But I think we will need to test with SSL or TLS, so here is another ticket https://fedorahosted.org/sssd/ticket/2820
Did I miss something?
LS
On Wed, Oct 07, 2015 at 08:58:32AM +0200, Lukas Slebodnik wrote:
On (31/08/15 12:31), Jakub Hrozek wrote:
On Fri, Aug 28, 2015 at 03:16:57PM +0200, Lukas Slebodnik wrote:
On (19/08/15 23:38), Jakub Hrozek wrote:
Hi,
as we're stabilizing the 1.13 branch and before we plan what we want to work on during the 1.14 development, we should use that time to write some more tests!
Here are some areas where we could add tests. Please discuss or add your ideas, I would like to turn this list into tickets we can start implementing:
- Extend the LDAP provider tests with more dynamic test cases.
- add a user to a group, run sss_cache, assert id user displays the new group and getent group displays the new member
- conversely with removing users from groups
There are some tests on list, but still it wuld be good to se a test plan.
- Background refresh
- could be built atop the LDAP NSS tests as well. I think we have all the infrastructure in place.
- Local overrides integration test:
- this could be relatively easy, just call the overrides tool and request the entry. Could be built atop the existing LDAP tests or even use the local provider.
Firstly, we need to prepare test cases for this feature. We can inspire in downstream ipa-views test case. After review of test cases we can split it to several tickets.
We already do: https://fedorahosted.org/sssd/ticket/2732
It 'just' needs an assignee..
IMHo this should be a very low priority ticket. It's already tested by downstream and they will have automated tests. We should focus to other parts which cannot be easily tested (whithout greping log files or touching/checking internals in sysdb) https://fedorahosted.org/sssd/ticket/2697
Yes, if Pavel didn't start on #2732 yet, then we can move it down, I agree PAM responder tests have bigger priority (also in Trac: major > minor), but..
BTW if the tests were ready together with feature it would be something different. It would not be a duplicated effort.
If we really want to duplicate effort then it would be better to see/review a test plan for local views.
..it's not completely useless though, we will refactor sssd_nss in 1.14, so we should have tests for views as well.
Transforming to pytest should be a trivial task. Should I file a ticket?
I'm not sure I understand? From the start I thought localviews tests would be in pytest, running tools and then getent..
- Add a KDC
- until we have a pam_wrapper, this would only be useful to test ldap_child, but adding the KDC instantiation might be worth it nonetheless
- there is a protorype of KDC instantiation on the list for some time now, since we enabled rootless SSSD
https://fedorahosted.org/sssd/ticket/2822
- IFP - could we reuse the existing sbus tests to spawn a custom bus?
Someone need to look into running dbus-daemon in cwrap env.
Then we need prepare test cases ...
Well, the first testcase could be to run introspection -- that would validate the bus is accessible.
It could be the first ticket, if you agree, I can file a ticket and assign it?
cwrap test for IFP should have the highest priority because it's not tested by downstream.
I think we should focus not only on what is untested, but what is untested and will be changed in the next release. Currently we have too much untested code :-) I know we will change the NSS responder and the LDAP provider so we need tests for those (we already have some for NSS).
So if we will change IFP in 1.14, then yes, otherwise I would prefer to add more LDAP tests and Samba first..
https://fedorahosted.org/sssd/ticket/2823 https://fedorahosted.org/sssd/ticket/2824 https://fedorahosted.org/sssd/ticket/2825
The first two tickets can be done in parallel.
- SUDO - can we trick sudo into connecting to our test sssd instance?
I thought it could be tested using pam_wrapper I filed a generat ticket for authentication + authorisation https://fedorahosted.org/sssd/ticket/2821
It can be split into testing pam provider and sudo provider.
I think the order of priority is roughly as above. I think the LDAP provider is critical enough to be well tested. The refresh and local override tests might be nice to have because we would be refactoring the NSS responder in 1.14, so we should have it tested.
I'll be happy to hear other opionions, though!
We can try to prepare fake AD in openldap (if possible) and test basic functionality with ldap provider.
I would like to talk to Andreas next week and ask about Samba. I hope this would be much easier and more robust.
Just for the record, here is a link to samba4-dc tracking ticket. https://fedorahosted.org/sssd/ticket/2818
But I think we will need to test with SSL or TLS, so here is another ticket https://fedorahosted.org/sssd/ticket/2820
Did I miss something?
I'm going to write tests for pam_sss.so as part of working on pam_wrapper, feel free to file a ticket and assign to me..
On 10/07/2015 10:23 AM, Jakub Hrozek wrote:
On Wed, Oct 07, 2015 at 08:58:32AM +0200, Lukas Slebodnik wrote:
On (31/08/15 12:31), Jakub Hrozek wrote:
On Fri, Aug 28, 2015 at 03:16:57PM +0200, Lukas Slebodnik wrote:
On (19/08/15 23:38), Jakub Hrozek wrote:
Hi,
as we're stabilizing the 1.13 branch and before we plan what we want to work on during the 1.14 development, we should use that time to write some more tests!
Here are some areas where we could add tests. Please discuss or add your ideas, I would like to turn this list into tickets we can start implementing:
- Extend the LDAP provider tests with more dynamic test cases.
- add a user to a group, run sss_cache, assert id user displays the new group and getent group displays the new member
- conversely with removing users from groups
There are some tests on list, but still it wuld be good to se a test plan.
- Background refresh
- could be built atop the LDAP NSS tests as well. I think we have all the infrastructure in place.
- Local overrides integration test:
- this could be relatively easy, just call the overrides tool and request the entry. Could be built atop the existing LDAP tests or even use the local provider.
Firstly, we need to prepare test cases for this feature. We can inspire in downstream ipa-views test case. After review of test cases we can split it to several tickets.
We already do: https://fedorahosted.org/sssd/ticket/2732
It 'just' needs an assignee..
IMHo this should be a very low priority ticket. It's already tested by downstream and they will have automated tests. We should focus to other parts which cannot be easily tested (whithout greping log files or touching/checking internals in sysdb) https://fedorahosted.org/sssd/ticket/2697
Yes, if Pavel didn't start on #2732 yet, then we can move it down, I agree PAM responder tests have bigger priority (also in Trac: major > minor), but..
BTW if the tests were ready together with feature it would be something different. It would not be a duplicated effort.
If we really want to duplicate effort then it would be better to see/review a test plan for local views.
..it's not completely useless though, we will refactor sssd_nss in 1.14, so we should have tests for views as well.
Transforming to pytest should be a trivial task. Should I file a ticket?
I'm not sure I understand? From the start I thought localviews tests would be in pytest, running tools and then getent..
Yes, that's how the test are. I'll send the first version on list today. I just need to resolve some final glitches.
On Wed, Oct 07, 2015 at 10:28:43AM +0200, Pavel Reichl wrote:
Yes, that's how the test are. I'll send the first version on list today. I just need to resolve some final glitches.
That's nice to hear that the tests will be added, but based on trac priorities I was expecting the PAM responder tests to arrive first..
On (07/10/15 10:23), Jakub Hrozek wrote:
On Wed, Oct 07, 2015 at 08:58:32AM +0200, Lukas Slebodnik wrote:
On (31/08/15 12:31), Jakub Hrozek wrote:
On Fri, Aug 28, 2015 at 03:16:57PM +0200, Lukas Slebodnik wrote:
On (19/08/15 23:38), Jakub Hrozek wrote:
Hi,
as we're stabilizing the 1.13 branch and before we plan what we want to work on during the 1.14 development, we should use that time to write some more tests!
Here are some areas where we could add tests. Please discuss or add your ideas, I would like to turn this list into tickets we can start implementing:
- Extend the LDAP provider tests with more dynamic test cases.
- add a user to a group, run sss_cache, assert id user displays the new group and getent group displays the new member
- conversely with removing users from groups
There are some tests on list, but still it wuld be good to se a test plan.
- Background refresh
- could be built atop the LDAP NSS tests as well. I think we have all the infrastructure in place.
- Local overrides integration test:
- this could be relatively easy, just call the overrides tool and request the entry. Could be built atop the existing LDAP tests or even use the local provider.
Firstly, we need to prepare test cases for this feature. We can inspire in downstream ipa-views test case. After review of test cases we can split it to several tickets.
We already do: https://fedorahosted.org/sssd/ticket/2732
It 'just' needs an assignee..
IMHo this should be a very low priority ticket. It's already tested by downstream and they will have automated tests. We should focus to other parts which cannot be easily tested (whithout greping log files or touching/checking internals in sysdb) https://fedorahosted.org/sssd/ticket/2697
Yes, if Pavel didn't start on #2732 yet, then we can move it down, I agree PAM responder tests have bigger priority (also in Trac: major > minor), but..
Both of these ticket are in state new.
BTW if the tests were ready together with feature it would be something different. It would not be a duplicated effort.
If we really want to duplicate effort then it would be better to see/review a test plan for local views.
..it's not completely useless though, we will refactor sssd_nss in 1.14, so we should have tests for views as well.
But it would be also tested by downstream so we could use their tests.
Transforming to pytest should be a trivial task. Should I file a ticket?
I'm not sure I understand? From the start I thought localviews tests would be in pytest, running tools and then getent..
I was writing about test plan for local views. It should be prepared/reviewed in advance. So should I file a ticket for preparing test plan? or do we want use test plan from downstream.
- Add a KDC
- until we have a pam_wrapper, this would only be useful to test ldap_child, but adding the KDC instantiation might be worth it nonetheless
- there is a protorype of KDC instantiation on the list for some time now, since we enabled rootless SSSD
https://fedorahosted.org/sssd/ticket/2822
- IFP - could we reuse the existing sbus tests to spawn a custom bus?
Someone need to look into running dbus-daemon in cwrap env.
Then we need prepare test cases ...
Well, the first testcase could be to run introspection -- that would validate the bus is accessible.
It could be the first ticket, if you agree, I can file a ticket and assign it?
cwrap test for IFP should have the highest priority because it's not tested by downstream.
I think we should focus not only on what is untested, but what is untested and will be changed in the next release. Currently we have too much untested code :-) I know we will change the NSS responder and the LDAP provider so we need tests for those (we already have some for NSS).
We do not have a lot of time for this. So we should not waste it. There is possibility to use downstream tests. So why should we duplicate effort. IFP is not tested at all. The tests for IFP could be used as an example/howto for other projects which would like to use IFP.
So if we will change IFP in 1.14, then yes, otherwise I would prefer to add more LDAP tests and Samba first..
If we have a time then we can implement additional tests for LDAP or Samba. We it's very likely it will take a long time to setup more samba-dc in cwrap. So +1 for sample test for samba, but not for full tests. But it would be better to focus on part which are not tested at all or for preparing sample test for each use case (setup samba4-dc, setup dbus-daemon, sample test for pam ...)
If sample tests are ready then additional test can be written as part of refactoring.
https://fedorahosted.org/sssd/ticket/2823 https://fedorahosted.org/sssd/ticket/2824 https://fedorahosted.org/sssd/ticket/2825
The first two tickets can be done in parallel.
- SUDO - can we trick sudo into connecting to our test sssd instance?
I thought it could be tested using pam_wrapper I filed a generat ticket for authentication + authorisation https://fedorahosted.org/sssd/ticket/2821
It can be split into testing pam provider and sudo provider.
I think the order of priority is roughly as above. I think the LDAP provider is critical enough to be well tested. The refresh and local override tests might be nice to have because we would be refactoring the NSS responder in 1.14, so we should have it tested.
I'll be happy to hear other opionions, though!
We can try to prepare fake AD in openldap (if possible) and test basic functionality with ldap provider.
I would like to talk to Andreas next week and ask about Samba. I hope this would be much easier and more robust.
Just for the record, here is a link to samba4-dc tracking ticket. https://fedorahosted.org/sssd/ticket/2818
But I think we will need to test with SSL or TLS, so here is another ticket https://fedorahosted.org/sssd/ticket/2820
Did I miss something?
I'm going to write tests for pam_sss.so as part of working on pam_wrapper, feel free to file a ticket and assign to me..
It's better if you assign yourself :-)
LS
On Wed, Oct 07, 2015 at 10:38:44AM +0200, Lukas Slebodnik wrote:
On (07/10/15 10:23), Jakub Hrozek wrote:
On Wed, Oct 07, 2015 at 08:58:32AM +0200, Lukas Slebodnik wrote:
On (31/08/15 12:31), Jakub Hrozek wrote:
On Fri, Aug 28, 2015 at 03:16:57PM +0200, Lukas Slebodnik wrote:
On (19/08/15 23:38), Jakub Hrozek wrote:
Hi,
as we're stabilizing the 1.13 branch and before we plan what we want to work on during the 1.14 development, we should use that time to write some more tests!
Here are some areas where we could add tests. Please discuss or add your ideas, I would like to turn this list into tickets we can start implementing:
- Extend the LDAP provider tests with more dynamic test cases.
- add a user to a group, run sss_cache, assert id user displays the new group and getent group displays the new member
- conversely with removing users from groups
There are some tests on list, but still it wuld be good to se a test plan.
- Background refresh
- could be built atop the LDAP NSS tests as well. I think we have all the infrastructure in place.
- Local overrides integration test:
- this could be relatively easy, just call the overrides tool and request the entry. Could be built atop the existing LDAP tests or even use the local provider.
Firstly, we need to prepare test cases for this feature. We can inspire in downstream ipa-views test case. After review of test cases we can split it to several tickets.
We already do: https://fedorahosted.org/sssd/ticket/2732
It 'just' needs an assignee..
IMHo this should be a very low priority ticket. It's already tested by downstream and they will have automated tests. We should focus to other parts which cannot be easily tested (whithout greping log files or touching/checking internals in sysdb) https://fedorahosted.org/sssd/ticket/2697
Yes, if Pavel didn't start on #2732 yet, then we can move it down, I agree PAM responder tests have bigger priority (also in Trac: major > minor), but..
Both of these ticket are in state new.
BTW if the tests were ready together with feature it would be something different. It would not be a duplicated effort.
If we really want to duplicate effort then it would be better to see/review a test plan for local views.
..it's not completely useless though, we will refactor sssd_nss in 1.14, so we should have tests for views as well.
But it would be also tested by downstream so we could use their tests.
Transforming to pytest should be a trivial task. Should I file a ticket?
I'm not sure I understand? From the start I thought localviews tests would be in pytest, running tools and then getent..
I was writing about test plan for local views. It should be prepared/reviewed in advance. So should I file a ticket for preparing test plan? or do we want use test plan from downstream.
We can base the tests on downstream.
- Add a KDC
- until we have a pam_wrapper, this would only be useful to test ldap_child, but adding the KDC instantiation might be worth it nonetheless
- there is a protorype of KDC instantiation on the list for some time now, since we enabled rootless SSSD
https://fedorahosted.org/sssd/ticket/2822
- IFP - could we reuse the existing sbus tests to spawn a custom bus?
Someone need to look into running dbus-daemon in cwrap env.
Then we need prepare test cases ...
Well, the first testcase could be to run introspection -- that would validate the bus is accessible.
It could be the first ticket, if you agree, I can file a ticket and assign it?
cwrap test for IFP should have the highest priority because it's not tested by downstream.
I think we should focus not only on what is untested, but what is untested and will be changed in the next release. Currently we have too much untested code :-) I know we will change the NSS responder and the LDAP provider so we need tests for those (we already have some for NSS).
We do not have a lot of time for this. So we should not waste it. There is possibility to use downstream tests. So why should we duplicate effort. IFP is not tested at all. The tests for IFP could be used as an example/howto for other projects which would like to use IFP.
Downstream tests are currently still not very easy to run. The point of the downstream tests is to be able to run "make check" or "make intgcheck" while working on code and see immediatelly if something is broken. The downstream tests don't have to have regression tests for all bugs and for all cases, but should cover the basics.
So if we will change IFP in 1.14, then yes, otherwise I would prefer to add more LDAP tests and Samba first..
If we have a time then we can implement additional tests for LDAP or Samba. We it's very likely it will take a long time to setup more samba-dc in cwrap. So +1 for sample test for samba, but not for full tests. But it would be better to focus on part which are not tested at all or for preparing sample test for each use case (setup samba4-dc, setup dbus-daemon, sample test for pam ...)
If sample tests are ready then additional test can be written as part of refactoring.
If we just add the very basic test (retrieve AD user) then we can add more tests as we change code in the AD provider..
I guess most of the work would be adding the plumbing, not the test anyway.
https://fedorahosted.org/sssd/ticket/2823 https://fedorahosted.org/sssd/ticket/2824 https://fedorahosted.org/sssd/ticket/2825
The first two tickets can be done in parallel.
- SUDO - can we trick sudo into connecting to our test sssd instance?
I thought it could be tested using pam_wrapper I filed a generat ticket for authentication + authorisation https://fedorahosted.org/sssd/ticket/2821
It can be split into testing pam provider and sudo provider.
I think the order of priority is roughly as above. I think the LDAP provider is critical enough to be well tested. The refresh and local override tests might be nice to have because we would be refactoring the NSS responder in 1.14, so we should have it tested.
I'll be happy to hear other opionions, though!
We can try to prepare fake AD in openldap (if possible) and test basic functionality with ldap provider.
I would like to talk to Andreas next week and ask about Samba. I hope this would be much easier and more robust.
Just for the record, here is a link to samba4-dc tracking ticket. https://fedorahosted.org/sssd/ticket/2818
But I think we will need to test with SSL or TLS, so here is another ticket https://fedorahosted.org/sssd/ticket/2820
Did I miss something?
I'm going to write tests for pam_sss.so as part of working on pam_wrapper, feel free to file a ticket and assign to me..
It's better if you assign yourself :-)
LS _______________________________________________ sssd-devel mailing list sssd-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/sssd-devel
On 10/07/2015 08:58 AM, Lukas Slebodnik wrote:
On (31/08/15 12:31), Jakub Hrozek wrote:
On Fri, Aug 28, 2015 at 03:16:57PM +0200, Lukas Slebodnik wrote:
On (19/08/15 23:38), Jakub Hrozek wrote:
Hi,
as we're stabilizing the 1.13 branch and before we plan what we want to work on during the 1.14 development, we should use that time to write some more tests!
Here are some areas where we could add tests. Please discuss or add your ideas, I would like to turn this list into tickets we can start implementing:
- Extend the LDAP provider tests with more dynamic test cases.
- add a user to a group, run sss_cache, assert id user displays the new group and getent group displays the new member
- conversely with removing users from groups
There are some tests on list, but still it wuld be good to se a test plan.
- Background refresh
- could be built atop the LDAP NSS tests as well. I think we have all the infrastructure in place.
- Local overrides integration test:
- this could be relatively easy, just call the overrides tool and request the entry. Could be built atop the existing LDAP tests or even use the local provider.
Firstly, we need to prepare test cases for this feature. We can inspire in downstream ipa-views test case. After review of test cases we can split it to several tickets.
We already do: https://fedorahosted.org/sssd/ticket/2732
It 'just' needs an assignee..
IMHo this should be a very low priority ticket. It's already tested by downstream and they will have automated tests. We should focus to other parts
Well our integration tests already found at least one real nasty bug in sss_override so I suppose that it's only good that the priority was higher and bug was found by us and not customer.
which cannot be easily tested (whithout greping log files or touching/checking internals in sysdb) https://fedorahosted.org/sssd/ticket/2697
BTW if the tests were ready together with feature it would be something different. It would not be a duplicated effort.
I'm not sure what you mean by duplicate effort but if you are referring to the fact that intg. tests were not prepared by the author of the feature then I have to disagree. I have found 2 documentation issues which would hardly by found by the author as he already know how the feature works. I believe that documentation issues are high priority as one of our major goals is to make SSSD more user friendly.
If we really want to duplicate effort then it would be better to see/review a test plan for local views. Transforming to pytest should be a trivial task. Should I file a ticket?
- Add a KDC
- until we have a pam_wrapper, this would only be useful to test ldap_child, but adding the KDC instantiation might be worth it nonetheless
- there is a protorype of KDC instantiation on the list for some time now, since we enabled rootless SSSD
https://fedorahosted.org/sssd/ticket/2822
- IFP - could we reuse the existing sbus tests to spawn a custom bus?
Someone need to look into running dbus-daemon in cwrap env.
Then we need prepare test cases ...
Well, the first testcase could be to run introspection -- that would validate the bus is accessible.
It could be the first ticket, if you agree, I can file a ticket and assign it?
cwrap test for IFP should have the highest priority because it's not tested by downstream.
https://fedorahosted.org/sssd/ticket/2823 https://fedorahosted.org/sssd/ticket/2824 https://fedorahosted.org/sssd/ticket/2825
The first two tickets can be done in parallel.
- SUDO - can we trick sudo into connecting to our test sssd instance?
I thought it could be tested using pam_wrapper I filed a generat ticket for authentication + authorisation https://fedorahosted.org/sssd/ticket/2821
It can be split into testing pam provider and sudo provider.
I think the order of priority is roughly as above. I think the LDAP provider is critical enough to be well tested. The refresh and local override tests might be nice to have because we would be refactoring the NSS responder in 1.14, so we should have it tested.
I'll be happy to hear other opionions, though!
We can try to prepare fake AD in openldap (if possible) and test basic functionality with ldap provider.
I would like to talk to Andreas next week and ask about Samba. I hope this would be much easier and more robust.
Just for the record, here is a link to samba4-dc tracking ticket. https://fedorahosted.org/sssd/ticket/2818
But I think we will need to test with SSL or TLS, so here is another ticket https://fedorahosted.org/sssd/ticket/2820
Did I miss something?
LS _______________________________________________ sssd-devel mailing list sssd-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/sssd-devel
On (07/10/15 17:42), Pavel Reichl wrote:
On 10/07/2015 08:58 AM, Lukas Slebodnik wrote:
On (31/08/15 12:31), Jakub Hrozek wrote:
On Fri, Aug 28, 2015 at 03:16:57PM +0200, Lukas Slebodnik wrote:
On (19/08/15 23:38), Jakub Hrozek wrote:
Hi,
as we're stabilizing the 1.13 branch and before we plan what we want to work on during the 1.14 development, we should use that time to write some more tests!
Here are some areas where we could add tests. Please discuss or add your ideas, I would like to turn this list into tickets we can start implementing:
- Extend the LDAP provider tests with more dynamic test cases.
- add a user to a group, run sss_cache, assert id user displays the new group and getent group displays the new member
- conversely with removing users from groups
There are some tests on list, but still it wuld be good to se a test plan.
- Background refresh
- could be built atop the LDAP NSS tests as well. I think we have all the infrastructure in place.
- Local overrides integration test:
- this could be relatively easy, just call the overrides tool and request the entry. Could be built atop the existing LDAP tests or even use the local provider.
Firstly, we need to prepare test cases for this feature. We can inspire in downstream ipa-views test case. After review of test cases we can split it to several tickets.
We already do: https://fedorahosted.org/sssd/ticket/2732
It 'just' needs an assignee..
IMHo this should be a very low priority ticket. It's already tested by downstream and they will have automated tests. We should focus to other parts
Well our integration tests already found at least one real nasty bug in sss_override so I suppose that it's only good that the priority was higher and bug was found by us and not customer.
which cannot be easily tested (whithout greping log files or touching/checking internals in sysdb) https://fedorahosted.org/sssd/ticket/2697
BTW if the tests were ready together with feature it would be something different. It would not be a duplicated effort.
I'm not sure what you mean by duplicate effort but if you are referring to the fact that intg. tests were not prepared by the author of the feature then
duplicate effor means when two persons do the same. Could you explain how is this related to the fact that tests were not prepared by author?
I have to disagree. I have found 2 documentation issues which would hardly by found by the author as he already know how the feature works. I believe that documentation issues are high priority as one of our major goals is to make SSSD more user friendly.
That's fine but sss_override would not be used by everyone. I would be curious how many bugs could be fould after extending PAM responder tests. The pam responder is used almost by everyone. (unless using just nss reponder). This might explain why the ticket for pam responder has higher priority. If you didn't agree about priority of test for loca override then we could discuss about it and then change. Priority of ticket should be accepted and not ignored. If everyone agree that priority of ticket is not useful then we can remove it from trac.
LS
On Thu, Oct 08, 2015 at 08:02:45AM +0200, Lukas Slebodnik wrote:
On (07/10/15 17:42), Pavel Reichl wrote:
On 10/07/2015 08:58 AM, Lukas Slebodnik wrote:
On (31/08/15 12:31), Jakub Hrozek wrote:
On Fri, Aug 28, 2015 at 03:16:57PM +0200, Lukas Slebodnik wrote:
On (19/08/15 23:38), Jakub Hrozek wrote:
Hi,
as we're stabilizing the 1.13 branch and before we plan what we want to work on during the 1.14 development, we should use that time to write some more tests!
Here are some areas where we could add tests. Please discuss or add your ideas, I would like to turn this list into tickets we can start implementing:
- Extend the LDAP provider tests with more dynamic test cases.
- add a user to a group, run sss_cache, assert id user displays the new group and getent group displays the new member
- conversely with removing users from groups
There are some tests on list, but still it wuld be good to se a test plan.
- Background refresh
- could be built atop the LDAP NSS tests as well. I think we have all the infrastructure in place.
- Local overrides integration test:
- this could be relatively easy, just call the overrides tool and request the entry. Could be built atop the existing LDAP tests or even use the local provider.
Firstly, we need to prepare test cases for this feature. We can inspire in downstream ipa-views test case. After review of test cases we can split it to several tickets.
We already do: https://fedorahosted.org/sssd/ticket/2732
It 'just' needs an assignee..
IMHo this should be a very low priority ticket. It's already tested by downstream and they will have automated tests. We should focus to other parts
Well our integration tests already found at least one real nasty bug in sss_override so I suppose that it's only good that the priority was higher and bug was found by us and not customer.
which cannot be easily tested (whithout greping log files or touching/checking internals in sysdb) https://fedorahosted.org/sssd/ticket/2697
BTW if the tests were ready together with feature it would be something different. It would not be a duplicated effort.
I'm not sure what you mean by duplicate effort but if you are referring to the fact that intg. tests were not prepared by the author of the feature then
duplicate effor means when two persons do the same. Could you explain how is this related to the fact that tests were not prepared by author?
1) I actually think tests not prepared by author are better, author might be coding in a 'tunnel vision' way, only touching the comfortable parts.
2) There was no time at the time we were writing the local overrides feature. Yes, it's bad that we completely underestimated the local overrides effort, but it happened and now we need to deal with it. And writing tests is a good way of dealing with no tests :-)
(I kinda agree about the ticket priority part, but I'm also happy we have more tests...the PAM responder tests will come next, I'm sure)
On 10/08/2015 08:02 AM, Lukas Slebodnik wrote:
On (07/10/15 17:42), Pavel Reichl wrote:
On 10/07/2015 08:58 AM, Lukas Slebodnik wrote:
On (31/08/15 12:31), Jakub Hrozek wrote:
On Fri, Aug 28, 2015 at 03:16:57PM +0200, Lukas Slebodnik wrote:
On (19/08/15 23:38), Jakub Hrozek wrote:
Hi,
as we're stabilizing the 1.13 branch and before we plan what we want to work on during the 1.14 development, we should use that time to write some more tests!
Here are some areas where we could add tests. Please discuss or add your ideas, I would like to turn this list into tickets we can start implementing:
- Extend the LDAP provider tests with more dynamic test cases.
- add a user to a group, run sss_cache, assert id user displays the new group and getent group displays the new member
- conversely with removing users from groups
There are some tests on list, but still it wuld be good to se a test plan.
- Background refresh
- could be built atop the LDAP NSS tests as well. I think we have all the infrastructure in place.
- Local overrides integration test:
- this could be relatively easy, just call the overrides tool and request the entry. Could be built atop the existing LDAP tests or even use the local provider.
Firstly, we need to prepare test cases for this feature. We can inspire in downstream ipa-views test case. After review of test cases we can split it to several tickets.
We already do: https://fedorahosted.org/sssd/ticket/2732
It 'just' needs an assignee..
IMHo this should be a very low priority ticket. It's already tested by downstream and they will have automated tests. We should focus to other parts
Well our integration tests already found at least one real nasty bug in sss_override so I suppose that it's only good that the priority was higher and bug was found by us and not customer.
which cannot be easily tested (whithout greping log files or touching/checking internals in sysdb) https://fedorahosted.org/sssd/ticket/2697
BTW if the tests were ready together with feature it would be something different. It would not be a duplicated effort.
I'm not sure what you mean by duplicate effort but if you are referring to the fact that intg. tests were not prepared by the author of the feature then
duplicate effor means when two persons do the same. Could you explain how is this related to the fact that tests were not prepared by author?
This is explained in the following paragraph.
I have to disagree. I have found 2 documentation issues which would hardly by found by the author as he already know how the feature works. I believe that documentation issues are high priority as one of our major goals is to make SSSD more user friendly.
That's fine but sss_override would not be used by everyone. I would be curious how many bugs could be fould after extending PAM responder tests. The pam responder is used almost by everyone.
The description of the referred ticket is: "Extend PAM responder unit test to cover code introduced in #1807([RFE] authenticate against cache in SSSD) This feature is definitely also not used by everyone.
(unless using just nss reponder). This might explain why the ticket for pam responder has higher priority. If you didn't agree about priority
You are right, priority of #2732 was lowered from major to minor.
of test for loca override then we could discuss about it and then change.
Yes, that would be better. On the other hand one of guidelines that our team should follow is: "Don't ask for permission ask for forgiveness". I did so and found real bugs that would be hit with customers. Unless there will be bugs that could be caught by unit test for "[RFE] authenticate against cache in SSSD" I don't think I have to ask for forgiveness.
Priority of ticket should be accepted and not ignored.
If everyone agree that priority of ticket is not useful then we can remove it from trac.
No need to overestimate this event.
LS _______________________________________________ sssd-devel mailing list sssd-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/sssd-devel
On Thu, Oct 08, 2015 at 11:02:39AM +0200, Pavel Reichl wrote:
Yes, that would be better. On the other hand one of guidelines that our team should follow is: "Don't ask for permission ask for forgiveness". I did so and found real bugs that would be hit with customers. Unless there will be bugs that could be caught by unit test for "[RFE] authenticate against cache in SSSD" I don't think I have to ask for forgiveness.
I don't think we need to discuss this anymore, just please keep in mind for next tickets, that's all.
It's also OK to disagree with ticket priority or ticket mileston -- again, it's not a big deal, just mark it as a higher priority. Others will see the notification and if there is any disagreement, we can discuss the change.
Most of all, I'm happy that we have tests and the tests found bugs.
sssd-devel@lists.fedorahosted.org