Re: RFC: -Wl,--as-needed by default (and glibc ldconfig file trigger)

[Tomasz Kłoczko]

So is it any final decision about start use by default --as-needed in
linker options?

Looking again on whole discussion across this thread and on
https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/3 I
don't see any arguments against start use --as-needed by default so
looks like only reason why such change still is not introduced is more
physiological than technical as such change will affect majority of
the packages in Fedora. bringing as result HUGE reduction of the
SONAME dependencies but at the same time still it may unhide some non
empty set of cases when still it will be necessary to apply some minor
fixes.
In my opinion number of affected packages will be very low (few).

BTW looks like similar situation like with --as-needed is with move
execution ldconfig from each package caring libraries to glibc file
trigger.
Ticket https://bugzilla.redhat.com/show_bug.cgi?id=1380878 which has
been opened more than year ago still has no go/no go decision and no
one is able to cast even single technical stone against such change.
As same as in case of use --as-neded most of the distribution packages
will be affected however in case of the ldconfig file trigger
introduction in the glibc it will change initially nothing so risk
here is even lower than in case of --as-needed (despite fact that even
here such risk is veeeery low). Simple introduction of the ldconfig
file trigger will unlock possibility to start developers remove
%post/%postub ldconfig execution one by one.

As f28 development cycle still is in the middle IMO it is right time
to apply at least one of those two changes (if not both).

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH

RFC: -Wl,--as-needed by default (and glibc ldconfig file trigger)

[Igor Gnatenko]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Tue, 2017-11-21 at 10:26 +0000, Tomasz Kłoczko wrote:
>  So is it any final decision about start use by default --as-needed
> in
>  linker options? 
Can you link Change Proposal you (or someone else) submitted? I have not heard
anything about that.
>  
>  Looking again on whole discussion across this thread and on
>  https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/3 I
>  don't see any arguments against start use --as-needed by default so
>  looks like only reason why such change still is not introduced is more
>  physiological than technical as such change will affect majority of
>  the packages in Fedora. bringing as result HUGE reduction of the
>  SONAME dependencies but at the same time still it may unhide some non
>  empty set of cases when still it will be necessary to apply some minor
>  fixes.
>  In my opinion number of affected packages will be very low (few). 
Counting numbers of affected packages by guessing is very bad idea.
>  
>  BTW looks like similar situation like with --as-needed is with move
>  execution ldconfig from each package caring libraries to glibc file
>  trigger.
>  Ticket https://bugzilla.redhat.com/show_bug.cgi?id=1380878 which has
>  been opened more than year ago still has no go/no go decision and no
>  one is able to cast even single technical stone against such change.
>  As same as in case of use --as-neded most of the distribution packages
>  will be affected however in case of the ldconfig file trigger
>  introduction in the glibc it will change initially nothing so risk
>  here is even lower than in case of --as-needed (despite fact that even
>  here such risk is veeeery low). Simple introduction of the ldconfig
>  file trigger will unlock possibility to start developers remove
>  %post/%postub ldconfig execution one by one.
>  
>  As f28 development cycle still is in the middle IMO it is right time
>  to apply at least one of those two changes (if not both). 
While personally I think both changes are good to make, there seem to be some
concert from glibc developers so it would be better if you could discuss this
in bugreport with glibc maintainers and FPC members.

Also I think we should not mix those 2 things.
- -- 
- -Igor Gnatenko
> -----BEGIN PGP SIGNATURE-----
>
> iQIzBAEBCAAdFiEEhLFO09aHZVqO+CM6aVcUvRu8X0wFAloUA08ACgkQaVcUvRu8
> X0yl6xAAgFNpeTdAlalhi4tTtA2o0I8T5a7Rju5fMA3tvvw3Tdm5S5dRlA/ZRVR/
> lGVYwb4OsVaNgoI18uY7MV4+rV2bWafQZPSjAOTBhGpevdvmaDCuQLMXVCwqZ6We
> /n8sn8L/Pzxu3D+Mk+OT0YJDyADkvbOHdDb/Bqj4vLQFqee2BO5DV0GShlTJWJod
> fBAgRlssLIT7ZQZwLZZrcfOnk8bVhUKl0/2jvtv9cq11G8mmkXKJ4A55GK5WOY8z
> kC+yeWhjpHlueBna/jYYUM4Bc2jIWcVljWBfeq5qyIIouJCGdsRLEGqMWiNABMmZ
> 5wgE9/Z8xxEavQHdW3kBKnwQfRZFlmlZkUT4KUE0mJuE65n6ZAWXwK1Ti+9bZKKT
> FgmjzVTI2q0IGeWl+QIRxGx1Ra0ZpzTYLxqX+U+AyI8jkznjf62YiddRQOj4UyLq
> 7q0WQ1Md0vjQxycAZQ2hHNsjG4GZgQZxpmSrJ9zENRgmia2yXvDiYkd1C9wWXIZ0
> L5mFoZDO4hl1gId5ocL+//NocPK1alBqGcVKXKh74NVfQqStxAg1Bv+la1KrXcOm
> ZR+sQKhwY8xhICeeHDGdJBgTUH+5IHFPZQRke0IJDGjPV/yoUy7l2hNl2qGPfXMd
> UtSMyRUy9Bt6Pe9zGDtbdSSJtDXqOKlV022+EptYht91OwETmxM=
> =ty9w
> -----END PGP SIGNATURE-----

RFC: -Wl,--as-needed by default (and glibc ldconfig file trigger)

[Tomasz Kłoczko]

On 21 November 2017 at 10:43, Igor Gnatenko
<ignatenkobrain(a)fedoraproject.org&gt; wrote:
>  -----BEGIN PGP SIGNED MESSAGE-----
>  Hash: SHA256
>
>  On Tue, 2017-11-21 at 10:26 +0000, Tomasz Kłoczko wrote:
> > So is it any final decision about start use by default --as-needed in
> > linker options?
>
>  Can you link Change Proposal you (or someone else) submitted? I have not heard
>  anything about that. 
https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/3

[..]
> > In my opinion number of affected packages will be very low
> (few).
>
>  Counting numbers of affected packages by guessing is very bad idea. 
Call it educated guess if you want.
More than decade ago when I've been leading maintenance of PLD
distribution applying --as-needed on the scale of ~3.6k source
packages.
During introduction of such change none of the packages has been left
behind and ~hundredth packages have been fixed by add
<package>-as-needed.patch files.
Most of those patches have been pushed to original source trees.
After what happened in PLD for example SuSE added to own binutils package patch

--- ld/ldmain.c.orig    2016-01-27 13:44:14.092983985 +0100
+++ ld/ldmain.c 2016-01-27 13:44:37.389254054 +0100
@@ -286,6 +286,8 @@ main (int argc, char **argv)
 #ifdef DEFAULT_FLAG_COMPRESS_DEBUG
   link_info.compress_debug = COMPRESS_DEBUG_GABI_ZLIB;
 #endif
+  if (getenv ("SUSE_ASNEEDED") && atoi(getenv
("SUSE_ASNEEDED")) > 0)
+    input_flags.add_DT_NEEDED_for_regular = TRUE;

   ldfile_add_arch ("");
   emulation = get_emulation (argc, argv);

which is kind of JFDI/dirty hack to add --as-needed as default linker behavior.
Other distributions like Arch Linux are using --as-needed as well as
default liner options.
As result many more packages have been fixed and those fixes mostly
have been in meantime as well integrated into source tree.

PLD, SuSE and Arch are not only iterations done on full scale distributions.
In the past I saw many --as-needed fixes in Debian patches and now it
is hard to find those patches.
SuSE has now ~12k source packages.
AltLinux (almost 18 src.rpm) IIRC is using --as-needed as well. I'm
not sure about Mageia (+20.5k src.rpm) .. I wouldn't be surprised if
Fedora is one of those few which are still not using --as-needed.

Do you really need more?

Many projects on switching from autoonf/automake/libtool to meson have
been test as meson adds --as-needed to linker options so all packages
build using meson are free of those issues.
More than decade ago *all* KDE packages added in own build frameworks
use --as-needed.
People around the world wasted probably thousands man/hours to
implement use --as-needed in many packages (for example net-snmp has

[tkloczko@domek net-snmp-5.7.3]$ ./configure --help | grep need
  --disable-as-needed             Link libperl against applications rather

This autoconf switch has been added only because net-snmp maintainer
did not understood nature of some perl related issues which should not
be fixed in net-snmp (btw necessary fix in perl in mean time has been
integrated). There is hundreds of packages which are already adding
--as-needed to the linker options in own build frameworks.

Probably more than year/man of developers time has been wasted because
binutils ld still is not producing by default DSOs linked with *only*
libraries which are needed.
To be hones still I do not understand why --as-needed is the option
because default ld behavior is causing produce non-optimally linked
binaries on MASSIVE SCALE.
This option should be IMO removed as current --as-needed behavior
should be hardcoded into ld!!!
One line change in redhat-rpm-config package will fix all those issues
at least on the scale of Fedora distribution.

I'm trying to support push such change in Fedora as for me it is way
easier to discuss importance of --as-needed than discussing it with
binutils maintainers who in many cases spend many years maintaining
only binutils code and by this organically do not understand why ld
should always produce correctly linked binaries.

So this is why I'm 100% sure that in +99.9% Fedora most frequently
used packages it will be no issues related to start use --as-needed.
If anything still could require add (trivial) fixes will be amongst
those less frequently used.

[..]
> > As f28 development cycle still is in the middle IMO it is right
> time
> > to apply at least one of those two changes (if not both).
>
>  While personally I think both changes are good to make, there seem to be some
>  concert from glibc developers so it would be better if you could discuss this
>  in bugreport with glibc maintainers and FPC members. 
Looks like you did not check what is in the comments of the ticket
which I've posted
(https://bugzilla.redhat.com/show_bug.cgi?id=1380878).
In comments of this ticket is not only whole discussion but as well
concrete propositions about how to implement glibc file trigger (with
my personal proposal about how to implement those triggers in simplest
possible form).

>  Also I think we should not mix those 2 things. 
Yes they are however looks like both changes are sharing the same
non-technical reason why they are still not introduced despite very
big positive impact which they care separately (probably bigger than
many other large changes in Fedora  introduced in recent years).
This is only reason why I've decided to glue those tow things together.

To be hones I'm not expecting any technical discussions about those
two changes as whatever was to discuss from technical point of view
already have been disused in more than last few years or decade in
case of --as-needed.
As all has been discussed and everything is ready I'm only expecting
answer from Fedora Packaging Committee on the question "when?" and
maybe some comments about "why not now yet?" (it maybe already
scheduled some other large scale change in Fedora and here is some
collision ..??).

As I wrote IMO looks like reasons why those two changes still have not
been integrated *are the same* and are not technical.
I can only guess most of the Fedora maintainers/developers are usually
working on scale of the single packages or few of them.
There is only few Fedora guys able to work on scale of whole groups of
the packages and even less on scale whole distribution.
Most of the Fedora developers are enough skilled and even less enough
experienced and confined to support such changes.
As result whole situation now is like in old joke about angels
visiting inferno to check why furnaces stopped heating it up ..

In other words looks like both changes are stuck in Fedora decision
making process mixed with shortage of man/hours people able to
introduce those changes.
As result we should be not discussing anything related to ld
--as-neded or glibc ldconfig file triggers per se but how focus few
decision making people that here we are dealing with some Fedora
maintenance problems.

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH

RFC: -Wl,--as-needed by default (and glibc ldconfig file trigger)

[David Tardon]

Hello,

On Tue, Nov 21, 2017 at 01:54:13PM +0000, Tomasz Kłoczko wrote:
>  On 21 November 2017 at 10:43, Igor Gnatenko
>  <ignatenkobrain(a)fedoraproject.org&gt; wrote:
>  > -----BEGIN PGP SIGNED MESSAGE-----
>  > Hash: SHA256
>  >
>  > On Tue, 2017-11-21 at 10:26 +0000, Tomasz Kłoczko wrote:
>  >> So is it any final decision about start use by default --as-needed in
>  >> linker options?
>  >
>  > Can you link Change Proposal you (or someone else) submitted? I have not heard
>  > anything about that.
>  
>  https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/3
>  
>  [..]
>  >> In my opinion number of affected packages will be very low (few).
>  >
>  > Counting numbers of affected packages by guessing is very bad idea.
>  
>  Call it educated guess if you want. 
You know, there is a way to get more reliable data: do scratch builds of
all Fedora packages and analyze the results. Then we'd know _exactly_
how many packages would need to be patched.

D.

RFC: -Wl,--as-needed by default (and glibc ldconfig file trigger)

[Stephan Bergmann]

On 11/21/2017 04:12 PM, David Tardon wrote:
>  On Tue, Nov 21, 2017 at 01:54:13PM +0000, Tomasz Kłoczko wrote:
> > On 21 November 2017 at 10:43, Igor Gnatenko
> > <ignatenkobrain(a)fedoraproject.org&gt; wrote:
> >> -----BEGIN PGP SIGNED MESSAGE-----
> >> Hash: SHA256
> >>
> >> On Tue, 2017-11-21 at 10:26 +0000, Tomasz Kłoczko wrote:
> >>> So is it any final decision about start use by default --as-needed in
> >>> linker options?
> >>
> >> Can you link Change Proposal you (or someone else) submitted? I have not heard
> >> anything about that.
> >
> > https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/3
> >
> > [..]
> >>> In my opinion number of affected packages will be very low (few).
> >>
> >> Counting numbers of affected packages by guessing is very bad idea.
> >
> > Call it educated guess if you want.
>  
>  You know, there is a way to get more reliable data: do scratch builds of
>  all Fedora packages and analyze the results. Then we'd know _exactly_
>  how many packages would need to be patched. 
...depending on how thoroughly the analysis is done, as the breakage 
caused by an erroneous --as-needed might, at least in principle, only 
happen at runtime, rather than at build time.

RFC: -Wl,--as-needed by default (and glibc ldconfig file trigger)

[Daniel P. Berrange]

On Tue, Nov 21, 2017 at 05:22:32PM +0100, Stephan Bergmann wrote:
>  On 11/21/2017 04:12 PM, David Tardon wrote:
>  > On Tue, Nov 21, 2017 at 01:54:13PM +0000, Tomasz Kłoczko wrote:
>  > > On 21 November 2017 at 10:43, Igor Gnatenko
>  > > <ignatenkobrain(a)fedoraproject.org&gt; wrote:
>  > > > -----BEGIN PGP SIGNED MESSAGE-----
>  > > > Hash: SHA256
>  > > > 
>  > > > On Tue, 2017-11-21 at 10:26 +0000, Tomasz Kłoczko wrote:
>  > > > > So is it any final decision about start use by default --as-needed
> in
>  > > > > linker options?
>  > > > 
>  > > > Can you link Change Proposal you (or someone else) submitted? I have not
> heard
>  > > > anything about that.
>  > > 
>  > > https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/3
>  > > 
>  > > [..]
>  > > > > In my opinion number of affected packages will be very low (few).
>  > > > 
>  > > > Counting numbers of affected packages by guessing is very bad idea.
>  > > 
>  > > Call it educated guess if you want.
>  > 
>  > You know, there is a way to get more reliable data: do scratch builds of
>  > all Fedora packages and analyze the results. Then we'd know _exactly_
>  > how many packages would need to be patched.
>  
>  ...depending on how thoroughly the analysis is done, as the breakage caused
>  by an erroneous --as-needed might, at least in principle, only happen at
>  runtime, rather than at build time. 
Pretty much all software in Fedora is already shipped in other distros
which have taken this linking approach for a long time. So it is pretty
reasonable to expect these kind of bugs have been identified & fixed
by maintainers in other distros already. There may be edge cases remaining
which no user has yet managed to tickle, but that's true of many aspects
of Fedora. eg when we introduced the hardened build flags, there were
places where we wouldn't know about breakage until much later. Requiring
perfection before making this kind of change effectively means never doing
the change.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

RFC: -Wl,--as-needed by default (and glibc ldconfig file trigger)

[Tomasz Kłoczko]

On 21 November 2017 at 16:36, Daniel P. Berrange <berrange(a)redhat.com&gt; wrote:
[..]
> > > > > Counting numbers of affected packages by guessing
> is very bad idea.
> > > >
> > > > Call it educated guess if you want.
> > >
> > > You know, there is a way to get more reliable data: do scratch builds of
> > > all Fedora packages and analyze the results. Then we'd know _exactly_
> > > how many packages would need to be patched. 
Yes of courses. Alternatively quite precise data could be generated by:

a)  build modified fedora-rpm-macros package and after this will be
possible to rebuild all Fedora packages to check which one are
failing. Such list of failing packages it will be necessary to compare
with koji data to remove those which is already known that they are
failing and on left list will be necessary to do one by one analyse to
identify those which are failing by add --as-needed to the linker
options. Still this list could be not 100% accurate as all Fedora
packages are constantly changing. You can do this if you have time and
other resources.

b) it is possible as well to take all current binary packages to
extract all elf binaries. After such extraction is possible to feed
database about list of all used and provided SONAMEs, list of all
external public symbols used and provided by each elf binary. After
analyse those data will be be possible to say:
1) which one binaries are linked with to many SONAMEs
2) which one binaries are still using some symbols which are available
only on run-time stage by indirect linking with libraries which are
linked with to many other libraries (AKA underlinking)
3) such static analyse can catch all possible run-time symbols clashes

Such statistic analyser/processor maybe it would be good to have in
Fedora but it would require relatively big investment into development
to catch probably only few cases each year.
********
Such effort to develop new infrastructure is way bigger than just
start use --as-needed by default then rebuild everything and at the
end fix only few new failing packages affected by use indirect
run-time linking. *** This way is WAY EASIER. Full stop ***
********

> > ...depending on how thoroughly the analysis is done, as the
> breakage caused
> > by an erroneous --as-needed might, at least in principle, only happen at
> > runtime, rather than at build time.
>
>  Pretty much all software in Fedora is already shipped in other distros
>  which have taken this linking approach for a long time. So it is pretty
>  reasonable to expect these kind of bugs have been identified & fixed
>  by maintainers in other distros already. There may be edge cases remaining
>  which no user has yet managed to tickle, but that's true of many aspects
>  of Fedora. eg when we introduced the hardened build flags, there were
>  places where we wouldn't know about breakage until much later. Requiring
>  perfection before making this kind of change effectively means never doing
>  the change. 
Guys I'm really appreciate that you are still thinking about some
technical aspects or other things not related straight to Fedora.
As I wrote (IMO) now main obstacle is not technical and it is more
about "which buttons needs to be pushed in Fedora to make it happen?"
Looks like no one want to put wood under this furnace and this is why
this inferno^KFedora packages have tenths of thousands bogus SONAME
dependencies still dangling between the packages.
When this will be implemented in Fedora maybe as well it will be
easier to convince binutils maintains to remove all --as-needed
optional code and leave this option as kind of void option.

And again: the same issue is with ldconfig file trigger which needs to
be added to glibc.
This few lines glibc spec improvement can reduce install or upgrade
packages time by at least 20% (less memory on the system or slower
storage than impact will be even bigger) by not wasting time on
ldconfig multiple time executing in every packages with libraries
scriptlets.
This change as well is about regain a lot of time and IOs on Fedora
build infrastructure.

kloczek
-- 
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH

RFC: -Wl,--as-needed by default (and glibc ldconfig file trigger)

[Igor Gnatenko]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Tue, 2017-11-21 at 18:09 +0000, Tomasz Kłoczko wrote:
>  On 21 November 2017 at 16:36, Daniel P. Berrange
> <berrange(a)redhat.com&gt; wrote:
>  [..]
>  > > > > > Counting numbers of affected packages by guessing is very bad
> idea.
>  > > > > 
>  > > > > Call it educated guess if you want.
>  > > > 
>  > > > You know, there is a way to get more reliable data: do scratch builds
>  > > > of
>  > > > all Fedora packages and analyze the results. Then we'd know _exactly_
>  > > > how many packages would need to be patched.
>  
>  Yes of courses. Alternatively quite precise data could be generated by:
>  
>  a)  build modified fedora-rpm-macros package and after this will be
>  possible to rebuild all Fedora packages to check which one are
>  failing. Such list of failing packages it will be necessary to compare
>  with koji data to remove those which is already known that they are
>  failing and on left list will be necessary to do one by one analyse to
>  identify those which are failing by add --as-needed to the linker
>  options. Still this list could be not 100% accurate as all Fedora
>  packages are constantly changing. You can do this if you have time and
>  other resources.
>  
>  b) it is possible as well to take all current binary packages to
>  extract all elf binaries. After such extraction is possible to feed
>  database about list of all used and provided SONAMEs, list of all
>  external public symbols used and provided by each elf binary. After
>  analyse those data will be be possible to say:
>  1) which one binaries are linked with to many SONAMEs
>  2) which one binaries are still using some symbols which are available
>  only on run-time stage by indirect linking with libraries which are
>  linked with to many other libraries (AKA underlinking)
>  3) such static analyse can catch all possible run-time symbols clashes
>  
>  Such statistic analyser/processor maybe it would be good to have in
>  Fedora but it would require relatively big investment into development
>  to catch probably only few cases each year.
>  ********
>  Such effort to develop new infrastructure is way bigger than just
>  start use --as-needed by default then rebuild everything and at the
>  end fix only few new failing packages affected by use indirect
>  run-time linking. *** This way is WAY EASIER. Full stop ***
>  ********
>  
>  > > ...depending on how thoroughly the analysis is done, as the breakage
>  > > caused
>  > > by an erroneous --as-needed might, at least in principle, only happen at
>  > > runtime, rather than at build time.
>  > 
>  > Pretty much all software in Fedora is already shipped in other distros
>  > which have taken this linking approach for a long time. So it is pretty
>  > reasonable to expect these kind of bugs have been identified & fixed
>  > by maintainers in other distros already. There may be edge cases remaining
>  > which no user has yet managed to tickle, but that's true of many aspects
>  > of Fedora. eg when we introduced the hardened build flags, there were
>  > places where we wouldn't know about breakage until much later. Requiring
>  > perfection before making this kind of change effectively means never doing
>  > the change.
>  
>  Guys I'm really appreciate that you are still thinking about some
>  technical aspects or other things not related straight to Fedora.
>  As I wrote (IMO) now main obstacle is not technical and it is more
>  about "which buttons needs to be pushed in Fedora to make it happen?"
>  Looks like no one want to put wood under this furnace and this is why
>  this inferno^KFedora packages have tenths of thousands bogus SONAME
>  dependencies still dangling between the packages.
>  When this will be implemented in Fedora maybe as well it will be
>  easier to convince binutils maintains to remove all --as-needed
>  optional code and leave this option as kind of void option. 
You keep ignoring me saying that **someone** needs to go through Change
Proposal procedure.

https://fedoraproject.org/wiki/Changes/Policy
>  
>  And again: the same issue is with ldconfig file trigger which needs to
>  be added to glibc.
>  This few lines glibc spec improvement can reduce install or upgrade
>  packages time by at least 20% (less memory on the system or slower
>  storage than impact will be even bigger) by not wasting time on
>  ldconfig multiple time executing in every packages with libraries
>  scriptlets.
>  This change as well is about regain a lot of time and IOs on Fedora
>  build infrastructure. 
You can submit Change Proposal too and if FESCo will approve change and glibc
maintainers will not have something strongly against - this will happen.

I know, this is some piece of bureaucracy, but it's rather good than bad.
- -- 
- -Igor Gnatenko
> -----BEGIN PGP SIGNATURE-----
>
> iQIzBAEBCAAdFiEEhLFO09aHZVqO+CM6aVcUvRu8X0wFAloUcDQACgkQaVcUvRu8
> X0zMChAAukrKydOjxa0htwFPZ0Yt9wupIto/LNsCSqDZEBww+qfYF43mits1RIZX
> Cddj/iHfMhpFYEex58pTdxAN8Q4HrIDZqvGteyhrBcxKCzEOeal53Xtg9N26TZeA
> JB5m5ZLqMe2i0gdzMjTvzODWZ1c7GK5fRF8GpFCXrynLG9J00Pg2FwK5H/k7FglM
> jjcjfWzH8AieyzFyVuBVYRDHjbOve9fKyyx2JUMKDpmf8Q7CVVOLI3SENdh5x1NJ
> 6i0ia2b/Hl8crgIzDdfSoqcZ0HrLpEukEF9vlLR5rdxLArLD8ddxHGyuZv8MxgsE
> 4zqwnvXBsUzrHHka5ZwVc+tIw+TpH3s/OcjCqU0RUQzTpERyatCIPuhlUjCOmX8/
> Ei9ODaboLzsr/2IkpoOuXt5rqHA/RQPbYIY+zu/2qNc4QGavSw/gW6DUsViiyqUg
> VsHRAGg/Y1SmuE35CNDJaeZgUPkY0wPd6Em1S9Loa0ETdxBj1Q5slTgBMMeqgFbN
> hzF5VMvOoPZif3m0L9CB/ATltUwIJrZO/yn7xIb1EfNRQSUiE/IqF7TnbAj5nIyN
> 8P3JT26Y/H/cI7Fd/JLMQaTGrkZzdrX7NrkfJI/ghk1w8wPJeh1qUV9BGuHyv8Q0
> KyZxWgx9RDPF77RUfvLCZgqn7GQl/WroS01d53UATg8H6M6lQlE=
> =Hpk0
> -----END PGP SIGNATURE-----