Compile with -fno-omit-frame-pointer on x86_64?

[Owen Taylor]

Lack of decent profiling is a major problem for making our operating
system fast. By far the most effective of profiling is sampling profile
with callgraph information.

Soeren's comment from March:

 http://lwn.net/Articles/380582/

Basically summarizes the situation, and as far as I know nothing has
changed ... with default compilation options, getting callgraph
profiling on x86_64 really requires a DWARF unwinder in the kernel.
Which seems unlikely to happen.

As a developer, your options for profiling are:

 - Recompile everything you care about profiling 
   with -fno-omit-frame-pointer instead of using system packages.

 - Switch to i386

Even if the second was reasonable to ask of developers, it also makes it
really hard to help users with performance problems if they have to
reinstall their system to give you a profile.

So, I'd like to bring up the possibility of switching to compiling our
packages with -fno-omit-frame-pointer for x86_64. As Soeren says, x86_64
isn't register starved, so the performance penalty shouldn't be huge.
But I have no idea if it's 0.5% or 5%.

What aspects of performance do we care about?
How would we measure the performance impact of changing
  compilation flags?
What is the acceptable slowdown?

- Owen

(One downside of any slowdown is that if we take a 1% hit and do
performance work that makes the system 10% faster, then we look bad by
comparison with other Linux distributions who get the advantages of the
performance work but don't take the 1% hit. Still, we should do what has
the biggest net gain for our users, right?)