Hardened builds

[Jerry James]

On Thu, Feb 26, 2015 at 11:00 AM, Rex Dieter <rdieter at math.unl.edu> wrote:
> which package is this again?  I can try experimenting a bit.
>
> The one that worked for me was lightdm, fwiw.

Oops, sorry, got distracted.  It is polymake.  That package has
multiple problems.

First, it invokes undefined behavior in one bit of code.  That
happened to work out with gcc 4.x, but gcc 5.x compiles the code a bit
differently, resulting in a scrambled data structure that causes a
crash sometime later.  Tracing that down was sure a pain, but I
finally found it, and will definitely tell upstream to fix their code.

Second, polymake queries perl::Config for the flags perl was built
with, and perl was built with the hardening flags.  No wonder I
couldn't get rid of them.

Third, polymake has a number of plugins ("apps" in polymake parlance),
some of which depend on each other, but they are not linked to one
another.  The main application knows about the dependencies between
them and dlopens them in the correct order to avoid unresolved
symbols.  However, it also has a documentation building step, which
dlopens them one by one to extract documentation strings.  This is
fine, because no code in the plugins is actually executed.  At least,
it used to be fine back when RTLD_LAZY actually worked.  Now it causes
the documentation building step to fail due to unresolved symbols.
The question is whether I can fix this by altering polymake alone, or
whether a non-hardened perl is needed to make this work.  I'm still
experimenting to discover the answer to that question.  (Polymake
builds take awhile and gobble memory like it was doughnuts, so this
isn't a fast process.)
-- 
Jerry James
http://www.jamezone.org/