[389-devel] Proof of concept: mocking DS in lib389

[Rich Megginson]

On 10/26/2013 12:49 AM, Jan Rusnacko wrote:
> On 10/25/2013 11:00 PM, Rich Megginson wrote:
>> On 10/25/2013 01:36 PM, Jan Rusnacko wrote:
>>> Hello Roberto and Thierry,
>>>
>>> as I promised, I am sending you a proof-of-concept code that demonstrates, how
>>> we can mock DS in unit tests for library function (see attachment). You can run
>>> tests just by executing py.test in tests directory.
>>>
>>> Only 3 files are of interest here:
>>>
>>> lib389/dsmodules/repl.py - this is a Python module with functions - they expect
>>> DS instance as the first argument. Since they are functions, not methods, I can
>>> just mock DS and pass that fake one as the first argument to them in unit tests.
>>>
>>> tests/test_dsmodules/conftest.py - this file contains definition of mock DS
>>> class along with py.test fixture, that returns it.
>>>
>>> tests/test_dsmodules/test_repl.py - this contains unit tests for functions from
>>> repl.py.
>>>
>>> What I do is quite simple - I override ldapadd, ldapdelete .. methods of mock DS
>>> class, so that instead of sending command to real DS instance, they just store
>>> the data in 'dit' dictionary (which represents content stored in DS). This way,
>>> I can check that when I call e.g. function enable_changelog(..), in the end DS
>>> will have correct changelog entry.
>>>
>>> To put it very bluntly - enable_changelog(..) function just adds correct
>>> changelog entry to whatever is passed to it as the first argument. In unit
>>> tests, it is mock DS, otherwise it would be real DS class that sends real ldap
>>> commands to real DS instance behind.
>> def test_add_repl_manager(fake_ds_inst_with_repl):
>>      ds_inst = fake_ds_inst_with_repl
>>      ds_inst.repl.add_repl_manager("cn=replication manager, cn=config", "Secret123")
>>      assert ds_inst.dit["cn=replication manager, cn=config"]["userPassword"] ==
>> "Secret123"
>>      assert ds_inst.dit["cn=replication manager, cn=config"]["nsIdleTimeout"] == "0"
>>      assert ds_inst.dit["cn=replication manager, cn=config"]["cn"] ==
>> "replication manager"
>>
>> If you are using a real directory server instance, doing add_repl_manager() is
>> going to make a real LDAP ADD request, right?
> Correct. If you pass DS with real ldapadd method that makes real reqests, its
> going to use that.
>> Will it still update the ds_inst.dit dict?
> ds_inst.dit is updated in mocked ldapadd. So in real ldapadd, no.
>> Wouldn't you have to do a real LDAP Search request to get the
>> actual values?
> Yes, correct. ds_inst.dit[] .. call is specific to mocked DS.
>
> But you are right - I could add fake ldapsearch method, that would return
> entries from 'dit' dictionary and use that to retrieve entries from mocked DS.

Because, otherwise, you have separate tests for mock DS and real DS?  Or 
perhaps I'm missing something?

>>> Now I can successfully test that enable_changelog really works, without going
>>> into trouble defining DSInstance or ldap calls at all. Also, I believe this
>>> approach would work for 95% of all functions in lib389. Another benefit is that
>>> unit tests are much faster, than on real DS instance.
>>>
>>> Sidenote: even though everything is defined in separate namespace of 'repl'
>>> module as function, in runtime they can be used as normal methods of class
>>> DSInstance. That is handled by DSModuleProxy. We already went through this, but
>>> not with Roberto.
>>>
>>> Hopefully, now with some code in our hands, we will be able to understand each
>>> other on this 'mocking' issue and come to conclusions more quickly.
>>>
>>> Let me know what you think.
>>>
>>> Thank you,
>>> Jan