[389-devel] Prereview #569: examine replication code to reduce amount of stored state information

[thierry bordaz]

Hi Ludwig,

    Ticket #569 is very important rework and attempt to clarify what
    state information are needed to resolve MMR update.
    The design and implementation looks simpler to follow but one
    complexity of state information is about all possible corner cases.
    It is very difficult to evaluate all the impacts of such changes.
    Here are few remarks:

    Design:

        Purging is related to deleted values no longer useful. The
        benefit of purging is to reduce the lenght of the entry to write
        to disk. Purging values is fine, but I think we should also
        purge csn. In fact quite often csn string is longer than the
        value it is related to. We purge deleted values from their too
        old csn (and possibly from the value itself), but I think we
        should also purge present value from their too old csn (of
        course value here is to be kept). Reading the code I think it is
        done in entry_delete_present_values_wsi* but I think it could
        also be done in entry_add_present_values_wsi*.


        Is purging values/csn on min(maxCSN) safe if we have no
        mechanism to guarantee that the RUV contains the RUVElement of
        all the suppliers ? I think it is one of the reason of "little
        conservative" 'min(maxCSN) - purgedelay' . A possibility to
        address that is to implement a periodic dummy replicated update
        on all the suppliers, so that even a not updated suppliers sends
        its RUVElement to the others. Of course it help only in topology
        with all the suppliers having this feature.

        I agree with creating a dummy value to store the 'adcsn', in
        case an attribute has no present/nor deleted values.

        distinguished value is complex to handle. I think it should not
        be considered at all while processing the mods. After all the
        mods have been apply, then we can go throught the entry an
        'resurect' or create the missing RDN values.

        Your sentence "It inspects every value in the present and
        deleted valueset although they in most cases already have been
        touched and eventually modified. Handling this in a higher layer
        could avoid unnecessary traversals"
        I do not know how you address this concern but my understanding
        is that it is important to go through present/deleted valueset
        to try to shrink the csnset.

    Source:

        Code looks nice and I have very few remarks.
        line 463: it is looking like the attribute is normalized (dn)
        before being recorded. Does that mean that the server
        systematically changes the provided attribute values ?

        entry_add_present_values_wsi_single_valued: there is no checking
        that the attribute has several values. I guess it is done later
        with schema checking. Why not doing this checking here.

        line 792: valueset_purge is called with op csn not the purged
        csn (min(maxCSN))


best regards
thierry
On 03/20/2013 03:11 PM, Ludwig Krispenz wrote:
> Hi,
>
> I have been working on ticket #569 and reached a state where I would 
> like to get feedback. The latest update contains a link to a page on 
> the wiki and an attached patch file.
>
> Please review both and let me know where you think more 
> investigation/explanation is needed and what your concerns are.
>
> Thanks,
> Ludwig
>
>
> -------- Original Message --------
> Subject: 	Re: [389 Project] #569: examine replication code to reduce 
> amount of stored state information
> Date: 	Wed, 20 Mar 2013 14:03:01 -0000
> From: 	389 Project <389-trac at fedorahosted.org>
> Reply-To: 	nobody at fedoraproject.org
> To: 	undisclosed-recipients:;
>
>
>
> #569: examine replication code to reduce amount of stored state information
> ------------------------------------------+---------------------------
>          Reporter:  lkrispen               |          Owner:  lkrispen
>              Type:  task                   |         Status:  assigned
>          Priority:  major                  |      Milestone:  1.3.2
>         Component:  Replication - General  |        Version:  1.3.0
>        Resolution:                         |       Keywords:
>        Blocked By:                         |       Blocking:
>            Review:                         |  Ticket origin:  Community
> Red Hat Bugzilla:                         |       Screened:  1
> ------------------------------------------+---------------------------
>
> Comment (by lkrispen):
>
>   The current status of the document is here:
>   http://port389.org/wiki/Entry_State_Resolution
>
>   The current state of the implementation is in the attached patch file.
>
>   What still needs to be done:
>   - complete the test cases and create test scripts
>   - verify acceptance test results, some tests fail since they examine
>   deleted values and value csns based on the old algorithm, need to modify
>   these test cases.
>   - there is one test failing because server seems to crash, need to
>   investigate this
>   - code also contains fix for ticket #602 by setting subsequence numbers,
>   investigate if this is correct for all usages of csn_compare or if
>   csn_compare needs to be differentiated
>
> -- 
> Ticket URL:<https://fedorahosted.org/389/ticket/569#comment:6>
> 389 Project<http://port389.org>
> 389 Directory Server
>
>
>
>
> --
> 389-devel mailing list
> 389-devel at lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/389-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fedoraproject.org/pipermail/389-devel/attachments/20130326/7a8c4b0b/attachment.html>