Exec-shield and memory randomization

[Dave Roberts]

Can somebody please point me at a good resource for understanding how
exec-shield and memory randomization work?

Specifically, I'm trying to understand the randomize_va_space variable
in /proc/sys/kernel/.

Is randomize_va_space independent of /proc/sys/kernel/exec-shield? That
is, if I "echo 0 > /proc/sys/kernel/exec-shield", I would have thought
that would also turn off memory randomization, but that doesn't seem to
be the case. Are things supposed to work that way, or am I goofing
something up?

Finally, is there a way to disable randomization on a per-binary basis
(with an ELF flag or lack thereof)? I seem have a binary (sbcl) that
doesn't like memory randomization. Rather than turn it off globally, I'd
rather just mark the binary as incompatible with it.

Thanks,

[Arjan van de Ven]

setarch has an -R option to start the binary without randomisation.
> Finally, is there a way to disable randomization on a per-binary basis
> (with an ELF flag or lack thereof)? I seem have a binary (sbcl) that
> doesn't like memory randomization. Rather than turn it off globally, I'd
> rather just mark the binary as incompatible with it.

I wonder why the app breaks; randomisation doesn't do anything weird at
all, in fact even before this stuff got in there was some extend of
randomisation already.

[Jason L Tibbitts III]

> > > > > "AvdV" == Arjan van de Ven  writes:

AvdV> I wonder why the app breaks; randomisation doesn't do anything
AvdV> weird at all, in fact even before this stuff got in there was
AvdV> some extend of randomisation already.

Haven't Lisp systems historically been problematic with respect to any
changes in memory layout?

- J<

[Dave Roberts]

On Sat, 2005-07-30 at 10:29 -0500, Jason L Tibbitts III wrote:
> > > > > > "AvdV" == Arjan van de Ven  writes:
>
> AvdV> I wonder why the app breaks; randomisation doesn't do anything
> AvdV> weird at all, in fact even before this stuff got in there was
> AvdV> some extend of randomisation already.
>
> Haven't Lisp systems historically been problematic with respect to any
> changes in memory layout?

It seems like it. I don't know enough about SBCL internals (yet), to
know exactly why this it's tripping up SBCL. My hunch is that sbcl uses
certain memory areas for storing different types of data and can
therefore use some sort of pointer arithmetic on some bits of a pointer
to quickly determine what type is being pointed at. That's just a hunch,
however.

In general, most of the Lisp-like systems seem to want more control over
memory placements.

[Arjan van de Ven]

On Sat, Jul 30, 2005 at 10:29:28AM -0500, Jason L Tibbitts III wrote:
> > > > > > "AvdV" == Arjan van de Ven  writes:
>
> AvdV> I wonder why the app breaks; randomisation doesn't do anything
> AvdV> weird at all, in fact even before this stuff got in there was
> AvdV> some extend of randomisation already.
>
> Haven't Lisp systems historically been problematic with respect to any
> changes in memory layout?

this isn't a change really; eg what randomisation does can happen anyway if you
do say,
 yum update glibc

[Dave Roberts]

On Sat, 2005-07-30 at 08:35 -0400, Arjan van de Ven wrote:
> setarch has an -R option to start the binary without randomisation.

Thanks, Arjan. I appreciate your help. I have a few follow-up questions:

Does setarch set the info in a persistent fashion? I'm reading the man
page for it (under FC3) and I can't quite tell what it does (setarch in
general, but it also doesn't document the -R option, BTW).

Hmmm.... in fact, trying it now on FC3, it doesn't seem to support that
option. Is -R only in FC4 right now? If so, there's probably a need to
release another setarch that supports that option for FC3, since it
looks like the latest FC3 kernel (2.6.12-1.1372_FC3) has
randomize_va_space defaulting to 1.

Also, there's a strange thing that I noticed when trying to debug this a
couple days ago: for some reason, older binaries that I built 3 or 4
months ago seem to work fine, while newer binaries don't. Did something
in gcc change that makes the old ones work and the new ones not? (like
GCC setting some compatibility bit in the ELF header differently in
newer GCCs).

Finally, is randomize_va_space supposed to be controlled by exec-shield?
When debugging, I first set exec-shield to 0, but that didn't seem to
have an effect. It was only when randomize_va_space get set to 0 that
things started working. That it, they seem independent, but most of the
documentation on exec-shield I have seen seems to suggest that turning
off exec-shield should turn off just about everything and leave you with
a pretty standard system, ala the pre-exec-shield days. Is that no
longer true?

[Arjan van de Ven]

> . That it, they seem independent, but most of the
> documentation on exec-shield I have seen seems to suggest that turning
> off exec-shield should turn off just about everything and leave you with
> a pretty standard system, ala the pre-exec-shield days. Is that no
> longer true?

well.. randomisation is now merged upstream....

[Dave Roberts]

On Sun, 2005-07-31 at 19:46 +0200, Arjan van de Ven wrote:
> > . That it, they seem independent, but most of the
> > documentation on exec-shield I have seen seems to suggest that turning
> > off exec-shield should turn off just about everything and leave you with
> > a pretty standard system, ala the pre-exec-shield days. Is that no
> > longer true?
>
> well.. randomisation is now merged upstream....

I'm not sure I understand. So that means "yes, they are now
independent" ?

So assuming that's the case, what does the kernel look for in
determining whether to turn of randomization on a per-binary basis? In
reading some older materials (like last year's Security Enhancements in
Red Hat Enterprise Linux paper by Drepper), it looked like the presence
of an explicitly executable stack segment in the ELF binary would turn
off all the various exec-shield enhancements, including randomization.
I'm guessing that this is still true for exec-shield, but does anything
now control randomization?

Running readelf and looking at the stack segment shows:

[dave@linux ~]$ readelf -l /usr/bin/sbcl | fgrep STACK
  GNU_STACK      0x000000 0x00000000 0x00000000 0x00000 0x00000 RWE 0x4

which as I understood it means that the stack is being marked as
executable (the "E" in the "RWE" field, right?).

So shouldn't this binary not be getting randomized memory addresses in
any case?

In any case, sorry to be persistent about this stuff. I have no desire
to be a pest. If you can point me to any up-to-date docs on this stuff,
I'd be happy to RTFM. I have been searching for anything I can get my
hands on but have been generally unsuccessful. Everything I read seems
to predate the change of randomization being merged upstream and so
short of reading the patches all myself (which comes next, I suppose), I
haven't found anything particular authoritative about how this works. An
email from yourself would be worth its weight in gold (at least if you
printed it out ;-).

[Arjan van de Ven]

On Sun, Jul 31, 2005 at 04:27:16PM -0700, Dave Roberts wrote:
> On Sun, 2005-07-31 at 19:46 +0200, Arjan van de Ven wrote:
> > > . That it, they seem independent, but most of the
> > > documentation on exec-shield I have seen seems to suggest that turning
> > > off exec-shield should turn off just about everything and leave you with
> > > a pretty standard system, ala the pre-exec-shield days. Is that no
> > > longer true?
> >
> > well.. randomisation is now merged upstream....
>
> I'm not sure I understand. So that means "yes, they are now
> independent" ?
>
> So assuming that's the case, what does the kernel look for in
> determining whether to turn of randomization on a per-binary basis?

Nowhere.. it's on for everything. The theory is that randomisation doesn't
do anything "odd" at all that couldn't happen otherwise. (by for example
upgrading a few libraries or so)
So I'd like to get to the bottom of why this app is breaking

[Juho Snellman]

 wrote:
> Nowhere.. it's on for everything. The theory is that randomisation doesn't
> do anything "odd" at all that couldn't happen otherwise. (by for example
> upgrading a few libraries or so)

I believe brk randomization is a new effect.
> So I'd like to get to the bottom of why this app is breaking

SBCL needs to mmap lots of non-relocatable data into certain memory
spaces, and the randomized heap location can overlap with one of these
spaces. I'm sure that making such assumptions about the memory layout
is technically incorrect. Still, this is rather unfortunate as there
seems to be no reasonable way to get the ELF loader to reserve memory
ranges.

[Roland McGrath]

> Still, this is rather unfortunate as there seems to be no reasonable way
> to get the ELF loader to reserve memory ranges.

The specified way is to use a PT_LOAD segment with p_flags 0, yielding a
PROT_NONE mapping that your program can mmap over or munmap after startup.
If this does not work right, then please file a bug about it.

[Juho Snellman]

 wrote:
> > Still, this is rather unfortunate as there seems to be no reasonable way
> > to get the ELF loader to reserve memory ranges.
>
> The specified way is to use a PT_LOAD segment with p_flags 0, yielding a
> PROT_NONE mapping that your program can mmap over or munmap after startup.
> If this does not work right, then please file a bug about it.

I don't feel it's a reasonable solution. (But I recognize that others
might think I'm being unreasonable here).

* Linker scripts are too fragile. You can't specify "use the
    internal default script, but add this PHDR and this section", so
    you need to copy the default, munge it in strange ways to actually
    make it work, and then hope that the next glibc/binutils/etc
    update doesn't break the script. We've had problems like this 
    with the linker script on the platform where we're forced to use it
    for other reasons.

Case in point, the script you posted a couple of years ago at
    http://lists.gnu.org/archive/html/mit-scheme-devel/2003-11/msg00009.html
    doesn't work any more, but requires moving the "required" PHDR
    below the other two PT_LOADs, and adding a FILEHDR option to it.
    This seems like a bizarre change, but that's the only way the script
    would accept more than two PT_LOADs.

* @nobits PT_LOADed segments don't just reserve the address space,
    but also count as allocated space for overcommit purposes (with
    the default 0 in /proc/sys/vm/overcommit_memory) unlike memory
    allocated with mmap. (Apparently because the protection bits are
    discarded, http://bugzilla.kernel.org/show_bug.cgi?id=2255)

We need to reserve large spaces and also run on low-memory
    machines.

* Even if both of the above problems were to be fixed in the next
    release of FC, we'd still need to find a solution that also works
    on older Linux installations.

The other common suggestion of writing/borrowing a ld.so replacement
doesn't seem much more palatable.