handling a 'debug' or 'development' setting

[Bill Nottingham]

In initscripts recently, we added code that would conditionally
enable two forms of malloc debugging. [1]  While this build was
disabled shortly after it hit the build system (because it
broke the build system [2]), it raised a more general issue.

We should have a generic framework for running in a 'debug'
or 'development' mode that does extra debugging at the sake
of some minimal amount of performance.

For example, right now in the rawhide kernel we enable various
debug options, which are then disabled when we do final builds
for a release. Wouldn't it be better if these could be switched
on somehow at runtime based on configuration?

Simiarly, there may be debugging parameters already in /proc
or /sys that can be twiddled if we're willing to run in
a debug mode.

This leads to a few questions:

1) What can we reasonably set?
   
   Right now, there's MALLOC_CHECK_ and MALLOC_PERTURB. There
   should be more. Is there anything in the various desktops
   that is appropriate?

2) How best to deploy this?

   The current malloc checking is added in the initscripts package
   as a profile.d script, enabled based on a file in /etc/sysconfig.
   While it could be done similar to how the kernel's debug support
   is done (rawhide builds set the config file to be enabled,
   final builds do not), I'm wondering if there's a better way.
   
   For example, do we have a 'fedora-debug' package that contains
   all the relevant settings that can be set? Do we make this
   required by a particular rawhide package, but not by the release
   package?

Just looking for brainstorming ideas right now.

Bill