MALLOC_PERTURB_: everyone should set this envvar

[Jim Meyering]

I posted about MALLOC_PERTURB_ about a year ago,

    http://thread.gmane.org/gmane.linux.redhat.fedora.devel/132690

but it is clear that not everyone is setting the variable, so for those
who didn't take the time last year, or who are new to the subject,
do yourself a favor and set MALLOC_PERTURB_ to a value in 1..255
everywhere.

For those who can't be bothered to click the link, here's that post:

    ------------
    If you are into development on glibc-based systems
    and do not set MALLOC_PERTURB_ to a nonzero value, then you
    are missing an easy opportunity to detect subtle bugs early.

    Sure, you can use valgrind, and it will detect whatever a
    MALLOC_PERTURB_ setting would have caught, and more, but it's
    far more expensive and takes some effort, however minimal.

    If you use zsh or bash, put this in one of your startup files:

        # http://udrepper.livejournal.com/11429.html
        export MALLOC_PERTURB_=$(($RANDOM % 255 + 1))

    and remember that when you find surprising bugs, that others
    who are also running tests (but without MALLOC_PERTURB_)
    will not see the same failures.

    This is useful enough that it is worth considering for inclusion
    in /etc/profile.
    ------------

Why do I insist?

Here's a nice example: a month or so ago I was investigating
a build problem in libvirt and as part of that, ran "make check"
from the cloned source tree.  Imagine my surprise when one of
the tests failed.  Obviously, while it was failing for me, it
was not failing for the many people who build libvirt regularly,
so what was different here?  I had MALLOC_PERTURB_ set in my
environment and they did not.

The bug I uncovered was a heap corruptor that dated back to
libvirt-0.9.5:

    http://thread.gmane.org/gmane.comp.emulators.libvirt/56605

TL;DR: Add these lines to your ~/.bash_profile or ~/.zshenv:

    # http://udrepper.livejournal.com/11429.html
    export MALLOC_PERTURB_=$(($RANDOM % 255 + 1))

One caveat: this does induce a small performance penalty (usually
negligible), so when you're measuring performance, you may want
to turn it off.

[John Reiser]

>      # http://udrepper.livejournal.com/11429.html
>      export MALLOC_PERTURB_=$(($RANDOM % 255 + 1)) 
For supportability (and to help remind you to turn it off before
performance tests), then also consider adding a line such as:
     echo 1>&2 MALLOC_PERTURB_=$MALLOC_PERTURB_  " #
$HOME/.bash_profile"

Also inexpensive:
     export MALLOC_CHECK_=1  # 0: off; 1: announce on stderr; 2: abort
     echo 1>&2 MALLOC_CHECK_=$MALLOC_CHECK_  " # $HOME/.bash_profile"
which handles double free(), single-byte overruns, etc.

For "always on" detection of overlapping memcpy() at low cost,
then consider  http://bitwagon.com/glibc-memlap/glibc-memlap.html .

--

[Jim Meyering]

John Reiser wrote:
> >     # http://udrepper.livejournal.com/11429.html
> >     export MALLOC_PERTURB_=$(($RANDOM % 255 + 1))
>
>  For supportability (and to help remind you to turn it off before
>  performance tests), then also consider adding a line such as:
>       echo 1>&2 MALLOC_PERTURB_=$MALLOC_PERTURB_  " #
> $HOME/.bash_profile" 
Good idea.  Or have a weekly cron job send clue-bat mail to
yourself reminding you that you do have this envvar set.

For the first year or so during which I had this enabled,
when I'd hit a memory-related problem, I didn't immediately think,
"Oh! that may be because I've enabled MALLOC_PERTURB_", so the
above might well help, initially.

If you do add something like that, be sure to use a guard so
that it is printed only for interactive sessions, e.g.,

    test -n "$PS1" && echo 1>&2 MALLOC_PERTURB_=$MALLOC_PERTURB_

Once you've been graced with enough MALLOC_PERTURB_-induced
failures, you get used to it, always remember to include it in
bug reports,... then you can turn that off ;-)

[Bill Nottingham]

Jim Meyering (jim(a)meyering.net) said: 
>  I posted about MALLOC_PERTURB_ about a year ago,
>  
>      http://thread.gmane.org/gmane.linux.redhat.fedora.devel/132690
>  
>  but it is clear that not everyone is setting the variable, so for those
>  who didn't take the time last year, or who are new to the subject,
>  do yourself a favor and set MALLOC_PERTURB_ to a value in 1..255
>  everywhere. 
See the 'debugmode' package.

Bill