[PATCHES] Use entry USN when available

[Simo Sorce]

This set of patches allows SSSD to use the more reliable entryUSN
against FreeIPA (and USNchanged against AD) when performing
enumerations.

If entryUSN(USNchanged) is not detected as available by checking rootdse
for lastUSN(highestCommittedUSN) then we fall back to use
modifyTimestamp which is fine in non-multimaster setups.

This set comprises 4 patches.

1. pass sdap_id_ctx to sdap_id_op functions (needed later).

2. cleanup unused vars and functions about rootdse to avoid confusion

3. add connection checks to test if USNs are available.

4. Change the code around to use the best USN method available and
fallback to the previous modifyTimestamp if not.

These pacthes have been briefly tested against a FreeIPA server with
the entryUSN configuration patch (still unpushed upstream) and all seem
to working fine.

Simo.

[Sumit Bose]

On Fri, Nov 26, 2010 at 06:49:33PM -0500, Simo Sorce wrote:
> This set of patches allows SSSD to use the more reliable entryUSN
> against FreeIPA (and USNchanged against AD) when performing
> enumerations.
>
> If entryUSN(USNchanged) is not detected as available by checking rootdse
> for lastUSN(highestCommittedUSN) then we fall back to use
> modifyTimestamp which is fine in non-multimaster setups.
>
> This set comprises 4 patches.
>
> pass sdap_id_ctx to sdap_id_op functions (needed later).
>
> cleanup unused vars and functions about rootdse to avoid confusion
>
> add connection checks to test if USNs are available.
>
> Change the code around to use the best USN method available and
>
> fallback to the previous modifyTimestamp if not.
>
> These pacthes have been briefly tested against a FreeIPA server with
> the entryUSN configuration patch (still unpushed upstream) and all seem
> to working fine.

I agree with the patches, but I would like to ask to rename
max_user_value and max_group_value to something like max_user_usn and
max_group_usn, because I find the original names very missleading.

Maybe it is easier for the LDAP server to optimize a search filter with
(!(%s<=%s)) instead of (%s>=%s)(!(%s=%s)) ?

OpenLDAP users might be happy if sssd would be able to autodetect
OpenLDAP and use entryCSN. Do you think check for the OpenLDAProotDSE
objectclass would be sufficient here?

bye,
Sumit
> Simo.
>
> -- 
> Simo Sorce * Red Hat, Inc * New York

[Simo Sorce]

On Wed, 1 Dec 2010 12:38:52 +0100
Sumit Bose  wrote:
> On Fri, Nov 26, 2010 at 06:49:33PM -0500, Simo Sorce wrote:
> > This set of patches allows SSSD to use the more reliable entryUSN
> > against FreeIPA (and USNchanged against AD) when performing
> > enumerations.
> >
> > If entryUSN(USNchanged) is not detected as available by checking
> > rootdse for lastUSN(highestCommittedUSN) then we fall back to use
> > modifyTimestamp which is fine in non-multimaster setups.
> >
> > This set comprises 4 patches.
> >
> > pass sdap_id_ctx to sdap_id_op functions (needed later).
> >
> > cleanup unused vars and functions about rootdse to avoid
> >
> > confusion
> >
> > add connection checks to test if USNs are available.
> >
> > Change the code around to use the best USN method available and
> >
> > fallback to the previous modifyTimestamp if not.
> >
> > These pacthes have been briefly tested against a FreeIPA server with
> > the entryUSN configuration patch (still unpushed upstream) and all
> > seem to working fine.
>
> I agree with the patches, but I would like to ask to rename
> max_user_value and max_group_value to something like max_user_usn and
> max_group_usn, because I find the original names very missleading.

I use value because when USNs are not available that thing contains the
max modifyTimestamp, so I wanted to make sure people think about it
when they see it.
> Maybe it is easier for the LDAP server to optimize a search filter
> with (!(%s<=%s)) instead of (%s>=%s)(!(%s=%s)) ?

I haven't changed the original filters, I can create an additional
patch for this if you like.
> OpenLDAP users might be happy if sssd would be able to autodetect
> OpenLDAP and use entryCSN. Do you think check for the OpenLDAProotDSE
> objectclass would be sufficient here?

Yes, give me the details and I can add that easily.

Simo.

[Sumit Bose]

On Wed, Dec 01, 2010 at 10:54:05AM -0500, Simo Sorce wrote:
> On Wed, 1 Dec 2010 12:38:52 +0100
> Sumit Bose  wrote:
> > On Fri, Nov 26, 2010 at 06:49:33PM -0500, Simo Sorce wrote:
> > > This set of patches allows SSSD to use the more reliable entryUSN
> > > against FreeIPA (and USNchanged against AD) when performing
> > > enumerations.
> > >
> > > If entryUSN(USNchanged) is not detected as available by checking
> > > rootdse for lastUSN(highestCommittedUSN) then we fall back to use
> > > modifyTimestamp which is fine in non-multimaster setups.
> > >
> > > This set comprises 4 patches.
> > >
> > > pass sdap_id_ctx to sdap_id_op functions (needed later).
> > >
> > > cleanup unused vars and functions about rootdse to avoid
> > >
> > > confusion
> > >
> > > add connection checks to test if USNs are available.
> > >
> > > Change the code around to use the best USN method available and
> > >
> > > fallback to the previous modifyTimestamp if not.
> > >
> > > These pacthes have been briefly tested against a FreeIPA server with
> > > the entryUSN configuration patch (still unpushed upstream) and all
> > > seem to working fine.
> >
> > I agree with the patches, but I would like to ask to rename
> > max_user_value and max_group_value to something like max_user_usn and
> > max_group_usn, because I find the original names very missleading.
>
> I use value because when USNs are not available that thing contains the
> max modifyTimestamp, so I wanted to make sure people think about it
> when they see it.

ok, what about max_(user|group)_ref_value ? Maybe someone else has a
good idea, too? It is just that my first, second and third toughts when
reading max_user_value were not related to the USN or modifyTimestamps,
but to UIDs or usernames. But I agree, that max_group_usn, might be to
restricted here.
> > Maybe it is easier for the LDAP server to optimize a search filter
> > with (!(%s<=%s)) instead of (%s>=%s)(!(%s=%s)) ?
>
> I haven't changed the original filters, I can create an additional
> patch for this if you like.

I leave this up to you. This was more a question than a suggestion to
change it. I'm not sure how LDAP servers try to optimize this kind of
queries and just thought that a shorter condition might be easier to
optimize. On the other hand '(%s>=%s)(!(%s=%s))' might be easier to
understand for the human eye.
> > OpenLDAP users might be happy if sssd would be able to autodetect
> > OpenLDAP and use entryCSN. Do you think check for the OpenLDAProotDSE
> > objectclass would be sufficient here?
>
> Yes, give me the details and I can add that easily.

I think the only detail is the OpenLDAProotDSE objectclass. I haven't
found anything else in the OpenLDAP rootDSE which can help here.

bye,
Sumit
> Simo.
>
> -- 
> Simo Sorce * Red Hat, Inc * New York
> _______________________________________________
> sssd-devel mailing list
> [e-mail]
> https://fedorahosted.org/mailman/listinfo/sssd-devel

[Simo Sorce]

On Mon, 6 Dec 2010 14:57:33 +0100
Sumit Bose  wrote:
> > I use value because when USNs are not available that thing contains
> > the max modifyTimestamp, so I wanted to make sure people think
> > about it when they see it.
>
> ok, what about max_(user|group)_ref_value ? Maybe someone else has a
> good idea, too? It is just that my first, second and third toughts
> when reading max_user_value were not related to the USN or
> modifyTimestamps, but to UIDs or usernames. But I agree, that
> max_group_usn, might be to restricted here.

What does _ref_ stands for ? Refresh ?
max_user|group_refresh_val ?
> > > Maybe it is easier for the LDAP server to optimize a search filter
> > > with (!(%s<=%s)) instead of (%s>=%s)(!(%s=%s)) ?
> >
> > I haven't changed the original filters, I can create an additional
> > patch for this if you like.
>
> I leave this up to you. This was more a question than a suggestion to
> change it. I'm not sure how LDAP servers try to optimize this kind of
> queries and just thought that a shorter condition might be easier to
> optimize. On the other hand '(%s>=%s)(!(%s=%s))' might be easier to
> understand for the human eye.

My thinking is that the current form may be more efficient, depending
on how indexes and filter parsing are optimized by the server.
I'll leave it as is for now.
> > > OpenLDAP users might be happy if sssd would be able to autodetect
> > > OpenLDAP and use entryCSN. Do you think check for the
> > > OpenLDAProotDSE objectclass would be sufficient here?
> >
> > Yes, give me the details and I can add that easily.
>
> I think the only detail is the OpenLDAProotDSE objectclass. I haven't
> found anything else in the OpenLDAP rootDSE which can help here.

Which attribute does openLdap uses on entries ?
entryUSN or USNChanged ?

Simo.

[Sumit Bose]

On Mon, Dec 06, 2010 at 09:14:45AM -0500, Simo Sorce wrote:
> On Mon, 6 Dec 2010 14:57:33 +0100
> Sumit Bose  wrote:
> > > I use value because when USNs are not available that thing contains
> > > the max modifyTimestamp, so I wanted to make sure people think
> > > about it when they see it.
> >
> > ok, what about max_(user|group)_ref_value ? Maybe someone else has a
> > good idea, too? It is just that my first, second and third toughts
> > when reading max_user_value were not related to the USN or
> > modifyTimestamps, but to UIDs or usernames. But I agree, that
> > max_group_usn, might be to restricted here.
>
> What does _ref_ stands for ? Refresh ?
> max_user|group_refresh_val ?

ok, let's keep it as it is. If someone will find a more crisp name it
can be changed later.
> > > > Maybe it is easier for the LDAP server to optimize a search filter
> > > > with (!(%s<=%s)) instead of (%s>=%s)(!(%s=%s)) ?
> > >
> > > I haven't changed the original filters, I can create an additional
> > > patch for this if you like.
> >
> > I leave this up to you. This was more a question than a suggestion to
> > change it. I'm not sure how LDAP servers try to optimize this kind of
> > queries and just thought that a shorter condition might be easier to
> > optimize. On the other hand '(%s>=%s)(!(%s=%s))' might be easier to
> > understand for the human eye.
>
> My thinking is that the current form may be more efficient, depending
> on how indexes and filter parsing are optimized by the server.
> I'll leave it as is for now.

ok
> > > > OpenLDAP users might be happy if sssd would be able to autodetect
> > > > OpenLDAP and use entryCSN. Do you think check for the
> > > > OpenLDAProotDSE objectclass would be sufficient here?
> > >
> > > Yes, give me the details and I can add that easily.
> >
> > I think the only detail is the OpenLDAProotDSE objectclass. I haven't
> > found anything else in the OpenLDAP rootDSE which can help here.
>
> Which attribute does openLdap uses on entries ?
> entryUSN or USNChanged ?

entryCSN, see http://www.openldap.org/faq/data/cache/1145.html for
details.

bye,
Sumit
> Simo.
>
> -- 
> Simo Sorce * Red Hat, Inc * New York
> _______________________________________________
> sssd-devel mailing list
> [e-mail]
> https://fedorahosted.org/mailman/listinfo/sssd-devel

[Simo Sorce]

On Mon, 6 Dec 2010 15:29:44 +0100
Sumit Bose  wrote:
> > > > > OpenLDAP users might be happy if sssd would be able to
> > > > > autodetect OpenLDAP and use entryCSN. Do you think check for
> > > > > the OpenLDAProotDSE objectclass would be sufficient here?
> > > >
> > > > Yes, give me the details and I can add that easily.
> > >
> > > I think the only detail is the OpenLDAProotDSE objectclass. I
> > > haven't found anything else in the OpenLDAP rootDSE which can
> > > help here.
> >
> > Which attribute does openLdap uses on entries ?
> > entryUSN or USNChanged ?
>
> entryCSN, see http://www.openldap.org/faq/data/cache/1145.html for
> details.

It is not clear to me that entryCSN is server specific and always
orders new entries at top, the format is also quite complex.
It is also not clear if access to it is available to anonymous clients.

If you do not mind I would defer implementation of its support to a
separate ticket/patch.

Simo.

[Sumit Bose]

On Mon, Dec 06, 2010 at 10:29:02AM -0500, Simo Sorce wrote:
> On Mon, 6 Dec 2010 15:29:44 +0100
> Sumit Bose  wrote:
> > > > > > OpenLDAP users might be happy if sssd would be able to
> > > > > > autodetect OpenLDAP and use entryCSN. Do you think check for
> > > > > > the OpenLDAProotDSE objectclass would be sufficient here?
> > > > >
> > > > > Yes, give me the details and I can add that easily.
> > > >
> > > > I think the only detail is the OpenLDAProotDSE objectclass. I
> > > > haven't found anything else in the OpenLDAP rootDSE which can
> > > > help here.
> > >
> > > Which attribute does openLdap uses on entries ?
> > > entryUSN or USNChanged ?
> >
> > entryCSN, see http://www.openldap.org/faq/data/cache/1145.html for
> > details.
>
> It is not clear to me that entryCSN is server specific and always
> orders new entries at top, the format is also quite complex.
> It is also not clear if access to it is available to anonymous clients.

iirc entryCSN is used by OpenLDAP during replication so I would expect
that newer entries get "larger" entryCSNs.
> If you do not mind I would defer implementation of its support to a
> separate ticket/patch.

ok, let's move this into a new ticket. This means

ACK

to the current version of the patches.

bye,
Sumit
> Simo.
>
> -- 
> Simo Sorce * Red Hat, Inc * New York
> _______________________________________________
> sssd-devel mailing list
> [e-mail]
> https://fedorahosted.org/mailman/listinfo/sssd-devel

[Stephen Gallagher]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12/07/2010 04:34 AM, Sumit Bose wrote:
> On Mon, Dec 06, 2010 at 10:29:02AM -0500, Simo Sorce wrote:
> > On Mon, 6 Dec 2010 15:29:44 +0100
> > Sumit Bose  wrote:
> > > > > > > OpenLDAP users might be happy if sssd would be able to
> > > > > > > autodetect OpenLDAP and use entryCSN. Do you think check for
> > > > > > > the OpenLDAProotDSE objectclass would be sufficient here?
> > > > > >
> > > > > > Yes, give me the details and I can add that easily.
> > > > >
> > > > > I think the only detail is the OpenLDAProotDSE objectclass. I
> > > > > haven't found anything else in the OpenLDAP rootDSE which can
> > > > > help here.
> > > >
> > > > Which attribute does openLdap uses on entries ?
> > > > entryUSN or USNChanged ?
> > >
> > > entryCSN, see http://www.openldap.org/faq/data/cache/1145.html for
> > > details.
> >
> > It is not clear to me that entryCSN is server specific and always
> > orders new entries at top, the format is also quite complex.
> > It is also not clear if access to it is available to anonymous clients.
>
> iirc entryCSN is used by OpenLDAP during replication so I would expect
> that newer entries get "larger" entryCSNs.
> > If you do not mind I would defer implementation of its support to a
> > separate ticket/patch.
>
> ok, let's move this into a new ticket. This means
>
> ACK
>
> to the current version of the patches.

Nack: fails to compile

../src/providers/ldap/ldap_id.c: In function 'sdap_check_online_done':
../src/providers/ldap/ldap_id.c:676:5: error: implicit declaration of
function 'sdap_cli_connect_recv_ext'

- -- 
Stephen Gallagher
RHCE 804006346421761

Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkz+fL8ACgkQeiVVYja6o6N2CQCeI/HalQpuf5hYL/a+/F6nKd+c
> GKIAn3BYmt/rCKw+CsWgBeMnCA/u3OJH
> =ba3y
> -----END PGP SIGNATURE-----

[Simo Sorce]

On Tue, 07 Dec 2010 13:28:15 -0500
Stephen Gallagher  wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 12/07/2010 04:34 AM, Sumit Bose wrote:
> > On Mon, Dec 06, 2010 at 10:29:02AM -0500, Simo Sorce wrote:
> > > On Mon, 6 Dec 2010 15:29:44 +0100
> > > Sumit Bose  wrote:
> > > > > > >> OpenLDAP users might be happy if sssd would be able to
> > > > > > >> autodetect OpenLDAP and use entryCSN. Do you think check for
> > > > > > >> the OpenLDAProotDSE objectclass would be sufficient here?
> > > > > > >
> > > > > > > Yes, give me the details and I can add that easily.
> > > > > >
> > > > > > I think the only detail is the OpenLDAProotDSE objectclass. I
> > > > > > haven't found anything else in the OpenLDAP rootDSE which can
> > > > > > help here.
> > > > >
> > > > > Which attribute does openLdap uses on entries ?
> > > > > entryUSN or USNChanged ?
> > > >
> > > > entryCSN, see http://www.openldap.org/faq/data/cache/1145.html for
> > > > details.
> > >
> > > It is not clear to me that entryCSN is server specific and always
> > > orders new entries at top, the format is also quite complex.
> > > It is also not clear if access to it is available to anonymous
> > > clients.
> >
> > iirc entryCSN is used by OpenLDAP during replication so I would
> > expect that newer entries get "larger" entryCSNs.
> > > If you do not mind I would defer implementation of its support to a
> > > separate ticket/patch.
> >
> > ok, let's move this into a new ticket. This means
> >
> > ACK
> >
> > to the current version of the patches.
>
> Nack: fails to compile
>
> ../src/providers/ldap/ldap_id.c: In function 'sdap_check_online_done':
> ../src/providers/ldap/ldap_id.c:676:5: error: implicit declaration of
> function 'sdap_cli_connect_recv_ext'

Oops should have rebuilt from scratch when I finally merged
sdap_cli_connect_recv() and sdap_cli_connect_recv_ext().

Attached rebased patchset that compiles properly.

Simo.

[Stephen Gallagher]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12/07/2010 01:58 PM, Simo Sorce wrote:
> On Tue, 07 Dec 2010 13:28:15 -0500
> Stephen Gallagher  wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > On 12/07/2010 04:34 AM, Sumit Bose wrote:
> > > On Mon, Dec 06, 2010 at 10:29:02AM -0500, Simo Sorce wrote:
> > > > On Mon, 6 Dec 2010 15:29:44 +0100
> > > > Sumit Bose  wrote:
> > > > > > >>> OpenLDAP users might be happy if sssd would be able to
> > > > > > >>> autodetect OpenLDAP and use entryCSN. Do you think check for
> > > > > > >>> the OpenLDAProotDSE objectclass would be sufficient here?
> > > > > > >>
> > > > > > >> Yes, give me the details and I can add that easily.
> > > > > > >
> > > > > > > I think the only detail is the OpenLDAProotDSE objectclass. I
> > > > > > > haven't found anything else in the OpenLDAP rootDSE which can
> > > > > > > help here.
> > > > > >
> > > > > > Which attribute does openLdap uses on entries ?
> > > > > > entryUSN or USNChanged ?
> > > > >
> > > > > entryCSN, see http://www.openldap.org/faq/data/cache/1145.html for
> > > > > details.
> > > >
> > > > It is not clear to me that entryCSN is server specific and always
> > > > orders new entries at top, the format is also quite complex.
> > > > It is also not clear if access to it is available to anonymous
> > > > clients.
> > >
> > > iirc entryCSN is used by OpenLDAP during replication so I would
> > > expect that newer entries get "larger" entryCSNs.
> > > > If you do not mind I would defer implementation of its support to a
> > > > separate ticket/patch.
> > >
> > > ok, let's move this into a new ticket. This means
> > >
> > > ACK
> > >
> > > to the current version of the patches.
> >
> > Nack: fails to compile
> >
> > ../src/providers/ldap/ldap_id.c: In function 'sdap_check_online_done':
> > ../src/providers/ldap/ldap_id.c:676:5: error: implicit declaration of
> > function 'sdap_cli_connect_recv_ext'
>
> Oops should have rebuilt from scratch when I finally merged
> sdap_cli_connect_recv() and sdap_cli_connect_recv_ext().
>
> Attached rebased patchset that compiles properly.
>
> Simo.

Ack with a minor change (will fix on push).

../src/providers/ldap/ldap_id_enum.c: In function 'enum_users_send':
../src/providers/ldap/ldap_id_enum.c:420:17: warning: unused variable
'attr_name'

I will remove the unused variable before I push.

- -- 
Stephen Gallagher
RHCE 804006346421761

Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkz+r58ACgkQeiVVYja6o6PFuwCgpIIFyTejUvQOkg3bm9EPGzld
> +x0AnRa74fmmIp9GHDj3Quva4PEme6gj
> =rZos
> -----END PGP SIGNATURE-----

[Stephen Gallagher]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12/07/2010 05:05 PM, Stephen Gallagher wrote:
> On 12/07/2010 01:58 PM, Simo Sorce wrote:
> > On Tue, 07 Dec 2010 13:28:15 -0500
> > Stephen Gallagher  wrote:
> > > -----BEGIN PGP SIGNED MESSAGE-----
> > > Hash: SHA1
> > >
> > > On 12/07/2010 04:34 AM, Sumit Bose wrote:
> > > > On Mon, Dec 06, 2010 at 10:29:02AM -0500, Simo Sorce wrote:
> > > > > On Mon, 6 Dec 2010 15:29:44 +0100
> > > > > Sumit Bose  wrote:
> > > > > > >
> > > > > > >>>> OpenLDAP users might be happy if sssd would be able to
> > > > > > >>>> autodetect OpenLDAP and use entryCSN. Do you think check for
> > > > > > >>>> the OpenLDAProotDSE objectclass would be sufficient here?
> > > > > > >>>
> > > > > > >>> Yes, give me the details and I can add that easily.
> > > > > > >>
> > > > > > >> I think the only detail is the OpenLDAProotDSE objectclass. I
> > > > > > >> haven't found anything else in the OpenLDAP rootDSE which can
> > > > > > >> help here.
> > > > > > >
> > > > > > > Which attribute does openLdap uses on entries ?
> > > > > > > entryUSN or USNChanged ?
> > > > > >
> > > > > > entryCSN, see http://www.openldap.org/faq/data/cache/1145.html for
> > > > > > details.
> > > > >
> > > > > It is not clear to me that entryCSN is server specific and always
> > > > > orders new entries at top, the format is also quite complex.
> > > > > It is also not clear if access to it is available to anonymous
> > > > > clients.
> > > >
> > > > iirc entryCSN is used by OpenLDAP during replication so I would
> > > > expect that newer entries get "larger" entryCSNs.
> > > > > If you do not mind I would defer implementation of its support to a
> > > > > separate ticket/patch.
> > > >
> > > > ok, let's move this into a new ticket. This means
> > > >
> > > > ACK
> > > >
> > > > to the current version of the patches.
> > >
> > > Nack: fails to compile
> > >
> > > ../src/providers/ldap/ldap_id.c: In function 'sdap_check_online_done':
> > > ../src/providers/ldap/ldap_id.c:676:5: error: implicit declaration of
> > > function 'sdap_cli_connect_recv_ext'
> > Oops should have rebuilt from scratch when I finally merged
> > sdap_cli_connect_recv() and sdap_cli_connect_recv_ext().
> > Attached rebased patchset that compiles properly.
> > Simo.
>
> Ack with a minor change (will fix on push).
>
> ../src/providers/ldap/ldap_id_enum.c: In function 'enum_users_send':
> ../src/providers/ldap/ldap_id_enum.c:420:17: warning: unused variable
> 'attr_name'
>
> I will remove the unused variable before I push.

Pushed to master (with minor fix).

- -- 
Stephen Gallagher
RHCE 804006346421761

Delivering value year after year.
Red Hat ranks #1 in value among software vendors.
http://www.redhat.com/promo/vendor/
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkz+sLkACgkQeiVVYja6o6P/3QCdFYYbOvQamePH/2ZOZ3DL5fYp
> 41wAoIrfn3inM4N6yJkxdOHuWboLUOTA
> =QhCU
> -----END PGP SIGNATURE-----