Here are some numbers(attached) using nbench (run on a core 2 duo
T7400 with 4MB L2 Cache).
To summarize the results:
i686 vs i586 does add +2% performance
going to x86_64 you gain +10 % (or 9.1% from i686)
from -O2 to -O3 there is a 4.9% performance win on x86_64 and 3.7%*
(more than i686 vs i586) on x86.
So instead of changing arch and breaking compatibility we should
switch from -O2 to -O3
* tested with -march=i686 can rerun with -march=i586 if needed.
-Os is also included.