[deployment-guide] Applied Petr's changes to the OProfile chapter.
Jaromir Hradilek
jhradile at fedoraproject.org
Sun Sep 4 01:45:13 UTC 2011
commit 973a387e6a26f6db77ffff7271b0dbb7bd1b0443
Author: Jaromir Hradilek <jhradile at redhat.com>
Date: Sun Sep 4 03:37:37 2011 +0200
Applied Petr's changes to the OProfile chapter.
Thanks, Petr!
en-US/OProfile.xml | 520 +++++++++++++++++++++++++++++++++-------------------
1 files changed, 330 insertions(+), 190 deletions(-)
---
diff --git a/en-US/OProfile.xml b/en-US/OProfile.xml
index d09a595..47bc0e9 100644
--- a/en-US/OProfile.xml
+++ b/en-US/OProfile.xml
@@ -14,7 +14,7 @@
significance="normal">
<primary>OProfile</primary>
</indexterm>
- <para>OProfile is a low overhead, system-wide performance monitoring tool. It uses the performance monitoring hardware on the processor to retrieve information about the kernel and executables on the system, such as when memory is referenced, the number of L2 cache requests, and the number of hardware interrupts received. On a &MAJOROS; system, the <filename>oprofile</filename> RPM package must be installed to use this tool.</para>
+ <para>OProfile is a low overhead, system-wide performance monitoring tool. It uses the performance monitoring hardware on the processor to retrieve information about the kernel and executables on the system, such as when memory is referenced, the number of L2 cache requests, and the number of hardware interrupts received. On a &MAJOROS; system, the <filename>oprofile</filename> package must be installed to use this tool.</para>
<para>Many processors include dedicated performance monitoring hardware. This hardware makes it possible to detect when certain events happen (such as the requested data not being in cache). The hardware normally takes the form of one or more <firstterm>counters</firstterm> that are incremented each time an event takes place. When the counter value, essentially rolls over, an interrupt is generated, making it possible to control the amount of detail (and therefore, overhead) produced by performance monitoring.</para>
<para>OProfile uses this hardware (or a timer-based substitute in cases where performance monitoring hardware is not present) to collect <firstterm>samples</firstterm> of performance-related data each time a counter generates an interrupt. These samples are periodically written out to disk; later, the data contained in these samples can then be used to generate reports on system-level and application-level performance.</para>
<para>OProfile is a useful tool, but be aware of some limitations when using it:</para>
@@ -30,7 +30,7 @@
<listitem>
<para>
<emphasis>
- <command>opreport</command> does not associate samples for inline functions' properly</emphasis> — <command>opreport</command> uses a simple address range mechanism to determine which function an address is in. Inline function samples are not attributed to the inline function but rather to the function the inline function was inserted into.</para>
+ <command>opreport</command> does not associate samples for inline functions properly</emphasis> — <command>opreport</command> uses a simple address range mechanism to determine which function an address is in. Inline function samples are not attributed to the inline function but rather to the function the inline function was inserted into.</para>
</listitem>
<listitem>
<para>
@@ -38,6 +38,10 @@
</listitem>
<listitem>
<para>
+ <emphasis>Hardware performance counters do not work on guest virtual machines</emphasis> — Because the hardware performance counters are not available on virtual systems, you need to use the <computeroutput>timer</computeroutput> mode. Run the command <command>opcontrol --deinit</command>, and then execute <command>modprobe oprofile timer=1</command> to enable the <computeroutput>timer</computeroutput> mode.</para>
+ </listitem>
+ <listitem>
+ <para>
<emphasis>Non-CPU-limited performance problems</emphasis> — OProfile is oriented to finding problems with CPU-limited processes. OProfile does not identify processes that are asleep because they are waiting on locks or for some other event to occur (for example an I/O device to finish an operation).</para>
</listitem>
</itemizedlist>
@@ -175,12 +179,10 @@
</tertiary>
</indexterm>
<!-- TBD6: /usr/lib/debug is an obsolete directory for Fedora 12 -->
- <screen>
-<command>opcontrol --setup --vmlinux=/usr/lib/debug/lib/modules/`uname -r`/vmlinux</command>
- </screen>
+ <screen>~]# <command>opcontrol --setup --vmlinux=/usr/lib/debug/lib/modules/`uname -r`/vmlinux</command></screen>
<note>
- <title>Note</title>
- <para>The <command>debuginfo</command> package must be installed (which contains the uncompressed kernel) in order to monitor the kernel.</para>
+ <title>Install the debuginfo package</title>
+ <para>The <package>debuginfo</package> package for the kernel must be installed (which contains the uncompressed kernel) in order to monitor the kernel.</para>
</note>
<para>To configure OProfile not to monitor the kernel, execute the following command as root:</para>
<indexterm
@@ -193,15 +195,9 @@
<option>--no-vmlinux</option>
</tertiary>
</indexterm>
- <screen>
-<command>opcontrol --setup --no-vmlinux</command>
- </screen>
+ <screen>~]# <command>opcontrol --setup --no-vmlinux</command></screen>
<para>This command also loads the <computeroutput>oprofile</computeroutput> kernel module, if it is not already loaded, and creates the <filename>/dev/oprofile/</filename> directory, if it does not already exist. Refer to <xref
linkend="s1-oprofile-dev-oprofile"/> for details about this directory.</para>
- <note>
- <title>Note</title>
- <para>Even if OProfile is configured not to profile the kernel, the SMP kernel still must be running so that the <command>oprofile</command> module can be loaded from it.</para>
- </note>
<para>Setting whether samples should be collected within the kernel only changes what data is collected, not how or where the collected data is stored. To generate different sample files for the kernel and application libraries, refer to <xref
linkend="s2-oprofile-starting-separate"/>.</para>
</section>
@@ -249,54 +245,54 @@
<tbody>
<row>
<entry>
- Pentium Pro
+ AMD64
</entry>
<entry>
- i386/ppro
+ x86-64/hammer
</entry>
<entry>
- 2
+ 4
</entry>
</row>
<row>
<entry>
- Pentium II
+ AMD Athlon
</entry>
<entry>
- i386/pii
+ i386/athlon
</entry>
<entry>
- 2
+ 4
</entry>
</row>
<row>
<entry>
- Pentium III
+ AMD Family 10h
</entry>
<entry>
- i386/piii
+ x86-64/family10
</entry>
<entry>
- 2
+ 4
</entry>
</row>
<row>
<entry>
- Pentium 4 (non-hyper-threaded)
+ AMD Family 11h
</entry>
<entry>
- i386/p4
+ x86-64/family11
</entry>
<entry>
- 8
+ 4
</entry>
</row>
<row>
<entry>
- Pentium 4 (hyper-threaded)
+ AMD Family 12h
</entry>
<entry>
- i386/p4-ht
+ x86-64/family12
</entry>
<entry>
4
@@ -304,10 +300,10 @@
</row>
<row>
<entry>
- Athlon
+ AMD Family 14h
</entry>
<entry>
- i386/athlon
+ x86-64/family14
</entry>
<entry>
4
@@ -315,18 +311,18 @@
</row>
<row>
<entry>
- AMD64
+ AMD Family 15h
</entry>
<entry>
- x86-64/hammer
+ x86-64/family15
</entry>
<entry>
- 4
+ 6
</entry>
</row>
<row>
<entry>
- TIMER_INT
+ IBM eServer System i and IBM eServer System p
</entry>
<entry>
timer
@@ -337,7 +333,40 @@
</row>
<row>
<entry>
- IBM eServer iSeries and pSeries
+ IBM POWER4
+ </entry>
+ <entry>
+ ppc64/power4
+ </entry>
+ <entry>
+ 8
+ </entry>
+ </row>
+ <row>
+ <entry>
+ IBM POWER5
+ </entry>
+ <entry>
+ ppc64/power5
+ </entry>
+ <entry>
+ 6
+ </entry>
+ </row>
+ <row>
+ <entry>
+ IBM PowerPC 970
+ </entry>
+ <entry>
+ ppc64/970
+ </entry>
+ <entry>
+ 8
+ </entry>
+ </row>
+ <row>
+ <entry>
+ IBM S/390 and IBM System z
</entry>
<entry>
timer
@@ -347,27 +376,33 @@
</entry>
</row>
<row>
- <entry></entry>
<entry>
- ppc64/power4
+ Intel Core i7
</entry>
<entry>
- 8
+ i386/core_i7
+ </entry>
+ <entry>
+ 4
</entry>
</row>
<row>
- <entry></entry>
<entry>
- ppc64/power5
+ Intel Nehalem microarchitecture
</entry>
<entry>
- 6
+ i386/nehalem
+ </entry>
+ <entry>
+ 4
</entry>
</row>
<row>
- <entry></entry>
<entry>
- ppc64/970
+ Intel Pentium 4 (non-hyper-threaded)
+ </entry>
+ <entry>
+ i386/p4
</entry>
<entry>
8
@@ -375,18 +410,29 @@
</row>
<row>
<entry>
- IBM eServer S/390 and S/390x
+ Intel Pentium 4 (hyper-threaded)
</entry>
<entry>
- timer
+ i386/p4-ht
</entry>
<entry>
- 1
+ 4
+ </entry>
+ </row>
+ <row>
+ <entry>
+ Intel Westmere microarchitecture
+ </entry>
+ <entry>
+ i386/westmere
+ </entry>
+ <entry>
+ 4
</entry>
</row>
<row>
<entry>
- IBM eServer zSeries
+ TIMER_INT
</entry>
<entry>
timer
@@ -436,7 +482,7 @@
<tbody>
<row>
<entry>
- Pentium Pro, Pentium II, Pentium III, Athlon, AMD64
+ AMD Athlon and AMD64
</entry>
<entry>
CPU_CLK_UNHALTED
@@ -447,29 +493,29 @@
</row>
<row>
<entry>
- Pentium 4 (HT and non-HT)
+ AMD Family 10h, AMD Family 11h, AMD Family 12h
</entry>
<entry>
- GLOBAL_POWER_EVENTS
+ CPU_CLK_UNHALTED
</entry>
<entry>
- The time during which the processor is not stopped
+ The processor's clock is not halted
</entry>
</row>
<row>
<entry>
- TIMER_INT
+ AMD Family 14h, AMD Family 15h
</entry>
<entry>
- (none)
+ CPU_CLK_UNHALTED
</entry>
<entry>
- Sample for each timer interrupt
+ The processor's clock is not halted
</entry>
</row>
<row>
<entry>
- ppc64/power4
+ IBM POWER4
</entry>
<entry>
CYCLES
@@ -480,7 +526,7 @@
</row>
<row>
<entry>
- ppc64/power5
+ IBM POWER5
</entry>
<entry>
CYCLES
@@ -491,7 +537,7 @@
</row>
<row>
<entry>
- ppc64/970
+ IBM PowerPC 970
</entry>
<entry>
CYCLES
@@ -500,13 +546,70 @@
Processor Cycles
</entry>
</row>
+
+
+
+
+ <row>
+ <entry>
+ Intel Core i7
+ </entry>
+ <entry>
+ CPU_CLK_UNHALTED
+ </entry>
+ <entry>
+ The processor's clock is not halted
+ </entry>
+ </row>
+ <row>
+ <entry>
+ Intel Nehalem microarchitecture
+ </entry>
+ <entry>
+ CPU_CLK_UNHALTED
+ </entry>
+ <entry>
+ The processor's clock is not halted
+ </entry>
+ </row>
+ <row>
+ <entry>
+ Intel Pentium 4 (hyper-threaded and non-hyper-threaded)
+ </entry>
+ <entry>
+ GLOBAL_POWER_EVENTS
+ </entry>
+ <entry>
+ The time during which the processor is not stopped
+ </entry>
+ </row>
+ <row>
+ <entry>
+ Intel Westmere microarchitecture
+ </entry>
+ <entry>
+ CPU_CLK_UNHALTED
+ </entry>
+ <entry>
+ The processor's clock is not halted
+ </entry>
+ </row>
+ <row>
+ <entry>
+ TIMER_INT
+ </entry>
+ <entry>
+ (none)
+ </entry>
+ <entry>
+ Sample for each timer interrupt
+ </entry>
+ </row>
</tbody>
</tgroup>
</table>
<para>The number of events that can be monitored at one time is determined by the number of counters for the processor. However, it is not a one-to-one correlation; on some processors, certain events must be mapped to specific counters. To determine the number of counters available, execute the following command:</para>
- <screen>
-<command>ls -d /dev/oprofile/[0-9]*</command>
- </screen>
+ <screen>~]# <command>ls -d /dev/oprofile/[0-9]*</command></screen>
<para>The events available vary depending on the processor type. To determine the events available for profiling, execute the following command as root (the list is specific to the system's processor type):</para>
<indexterm
significance="normal">
@@ -521,17 +624,12 @@
<command>ophelp</command>
</primary>
</indexterm>
- <screen>
-<command>ophelp</command>
- </screen>
+ <screen>~]# <command>ophelp</command></screen>
<para>The events for each counter can be configured via the command line or with a graphical interface. For more information on the graphical interface, refer to <xref
linkend="s1-oprofile-gui"/>. If the counter cannot be set to a specific event, an error message is displayed.</para>
<para>To set the event for each configurable counter via the command line, use <command>opcontrol</command>:</para>
- <screen>
-<command>opcontrol --event=<replaceable><event-name></replaceable>:<replaceable><sample-rate></replaceable>
- </command>
- </screen>
- <para>Replace <replaceable><event-name></replaceable> with the exact name of the event from <command>ophelp</command>, and replace <replaceable><sample-rate></replaceable> with the number of events between samples.</para>
+ <screen>~]# <command>opcontrol --event=<replaceable>event-name</replaceable>:<replaceable>sample-rate</replaceable></command></screen>
+ <para>Replace <replaceable>event-name</replaceable> with the exact name of the event from <command>ophelp</command>, and replace <replaceable>sample-rate</replaceable> with the number of events between samples.</para>
<section
id="s3-oprofile-events-sampling">
<title>Sampling Rate</title>
@@ -543,13 +641,10 @@
</indexterm>
<para>By default, a time-based event set is selected. It creates a sample every 100,000 clock cycles per processor. If the timer interrupt is used, the timer is set to whatever the jiffy rate is and is not user-settable. If the <computeroutput>cpu_type</computeroutput> is not <computeroutput>timer</computeroutput>, each event can have a <firstterm>sampling rate</firstterm> set for it. The sampling rate is the number of events between each sample snapshot.</para>
<para>When setting the event for the counter, a sample rate can also be specified:</para>
- <screen>
-<command>opcontrol --event=<replaceable><event-name></replaceable>:<replaceable><sample-rate></replaceable>
- </command>
- </screen>
- <para>Replace <replaceable><sample-rate></replaceable> with the number of events to wait before sampling again. The smaller the count, the more frequent the samples. For events that do not happen frequently, a lower count may be needed to capture the event instances.</para>
+ <screen>~]# <command>opcontrol --event=<replaceable>event-name</replaceable>:<replaceable>sample-rate</replaceable></command></screen>
+ <para>Replace <replaceable>sample-rate</replaceable> with the number of events to wait before sampling again. The smaller the count, the more frequent the samples. For events that do not happen frequently, a lower count may be needed to capture the event instances.</para>
<warning>
- <title>Caution</title>
+ <title>Sampling too frequently can overload the system</title>
<para>Be extremely careful when setting sampling rates. Sampling too frequently can overload the system, causing the system to appear as if it is frozen or causing the system to actually freeze.</para>
</warning>
</section>
@@ -563,10 +658,7 @@
</indexterm>
<para>Some user performance monitoring events may also require unit masks to further define the event.</para>
<para>Unit masks for each event are listed with the <command>ophelp</command> command. The values for each unit mask are listed in hexadecimal format. To specify more than one unit mask, the hexadecimal values must be combined using a bitwise <firstterm>or</firstterm> operation.</para>
- <screen>
-<command>opcontrol --event=<replaceable><event-name></replaceable>:<replaceable><sample-rate></replaceable>:<replaceable><unit-mask></replaceable>
- </command>
- </screen>
+ <screen>~]# <command>opcontrol --event=<replaceable>event-name</replaceable>:<replaceable>sample-rate</replaceable>:<replaceable>unit-mask</replaceable></command></screen>
</section>
</section>
<section
@@ -579,28 +671,17 @@
<tertiary>separating profiles</tertiary>
</indexterm>
<para>By default, kernel mode and user mode information is gathered for each event. To configure OProfile to ignore events in kernel mode for a specific counter, execute the following command:</para>
- <screen>
-<command>opcontrol --event=<replaceable><event-name></replaceable>:<replaceable><sample-rate></replaceable>:<replaceable><unit-mask></replaceable>:0</command>
- </screen>
+ <screen>~]# <command>opcontrol --event=<replaceable>event-name</replaceable>:<replaceable>sample-rate</replaceable>:<replaceable>unit-mask</replaceable>:0</command></screen>
<para>Execute the following command to start profiling kernel mode for the counter again:</para>
- <screen>
-<command>opcontrol --event=<replaceable><event-name></replaceable>:<replaceable><sample-rate></replaceable>:<replaceable><unit-mask></replaceable>:1</command>
- </screen>
+ <screen>~]# <command>opcontrol --event=<replaceable>event-name</replaceable>:<replaceable>sample-rate</replaceable>:<replaceable>unit-mask</replaceable>:1</command></screen>
<para>To configure OProfile to ignore events in user mode for a specific counter, execute the following command:</para>
- <screen>
-<command>opcontrol --event=<replaceable><event-name></replaceable>:<replaceable><sample-rate></replaceable>:<replaceable><unit-mask></replaceable>:<replaceable><kernel></replaceable>:0</command>
- </screen>
+ <screen>~]# <command>opcontrol --event=<replaceable>event-name</replaceable>:<replaceable>sample-rate</replaceable>:<replaceable>unit-mask</replaceable>:<replaceable>kernel</replaceable>:0</command></screen>
<para>Execute the following command to start profiling user mode for the counter again:</para>
- <screen>
-<command>opcontrol --event=<replaceable><event-name></replaceable>:<replaceable><sample-rate></replaceable>:<replaceable><unit-mask></replaceable>:<replaceable><kernel></replaceable>:1</command>
- </screen>
+ <screen>~]# <command>opcontrol --event=<replaceable>event-name</replaceable>:<replaceable>sample-rate</replaceable>:<replaceable>unit-mask</replaceable>:<replaceable>kernel</replaceable>:1</command></screen>
<para>When the OProfile daemon writes the profile data to sample files, it can separate the kernel and library profile data into separate sample files. To configure how the daemon writes to sample files, execute the following command as root:</para>
- <screen>
-<command>opcontrol --separate=<replaceable><choice></replaceable>
- </command>
- </screen>
+ <screen>~]# <command>opcontrol --separate=<replaceable>choice</replaceable></command></screen>
<para>
- <replaceable><choice></replaceable> can be one of the following:</para>
+ <replaceable>choice</replaceable> can be one of the following:</para>
<itemizedlist>
<listitem>
<para>
@@ -621,8 +702,8 @@
</itemizedlist>
<para>If <option>--separate=library</option> is used, the sample file name includes the name of the executable as well as the name of the library.</para>
<note>
- <title>Note</title>
- <para>These configuration changes will take effect when <command>oprofile</command> is restarted.</para>
+ <title>Restart the OProfile profiler</title>
+ <para>These configuration changes will take effect when the OProfile profiler is restarted.</para>
</note>
</section>
</section>
@@ -645,9 +726,7 @@
<option>--start</option>
</tertiary>
</indexterm>
- <screen>
-<command>opcontrol --start</command>
- </screen>
+ <screen>~]# <command>opcontrol --start</command></screen>
<para>Output similar to the following is displayed:</para>
<screen>Using log file /var/lib/oprofile/oprofiled.log Daemon started. Profiler running.</screen>
<para>The settings in <filename>/root/.oprofile/daemonrc</filename> are used.</para>
@@ -674,10 +753,16 @@
<tertiary>log file</tertiary>
</indexterm>
<para>The OProfile daemon, <command>oprofiled</command>, is started; it periodically writes the sample data to the <filename>/var/lib/oprofile/samples/</filename> directory. The log file for the daemon is located at <filename>/var/lib/oprofile/oprofiled.log</filename>.</para>
+ <important>
+ <title>Disable the nmi_watchdog registers</title>
+ <para>On a &MAJOROSVER; system, the <computeroutput>nmi_watchdog</computeroutput> registers with the <computeroutput>perf</computeroutput> subsystem. Due to this, the <computeroutput>perf</computeroutput> subsystem grabs control of the performance counter registers at boot time, blocking OProfile from working.</para>
+ <para>To resolve this, either boot with the <command>nmi_watchdog=0</command> kernel parameter set, or run the following command to disable <computeroutput>nmi_watchdog</computeroutput> at run time:</para>
+ <screen>~]# <command>echo 0 > /proc/sys/kernel/nmi_watchdog</command></screen>
+ <para>To re-enable <computeroutput>nmi_watchdog</computeroutput>, use the following command:</para>
+ <screen>~]# <command>echo 1 > /proc/sys/kernel/nmi_watchdog</command></screen>
+ </important>
<para>To stop the profiler, execute the following command as root:</para>
- <screen>
-<command>opcontrol --shutdown</command>
- </screen>
+ <screen>~]# <command>opcontrol --shutdown</command></screen>
</section>
<section
id="s1-oprofile-saving-data">
@@ -688,11 +773,8 @@
<secondary>saving data</secondary>
</indexterm>
<para>Sometimes it is useful to save samples at a specific time. For example, when profiling an executable, it may be useful to gather different samples based on different input data sets. If the number of events to be monitored exceeds the number of counters available for the processor, multiple runs of OProfile can be used to collect data, saving the sample data to different files each time.</para>
- <para>To save the current set of sample files, execute the following command, replacing <replaceable><name></replaceable> with a unique descriptive name for the current session.</para>
- <screen>
-<command>opcontrol --save=<replaceable><name></replaceable>
- </command>
- </screen>
+ <para>To save the current set of sample files, execute the following command, replacing <replaceable>name</replaceable> with a unique descriptive name for the current session.</para>
+ <screen>~]# <command>opcontrol --save=<replaceable>name</replaceable></command></screen>
<para>The directory <filename>/var/lib/oprofile/samples/<replaceable>name</replaceable>/</filename> is created and the current sample files are copied to it.</para>
</section>
<section
@@ -704,9 +786,7 @@
<secondary>reading data</secondary>
</indexterm>
<para>Periodically, the OProfile daemon, <command>oprofiled</command>, collects the samples and writes them to the <filename>/var/lib/oprofile/samples/</filename> directory. Before reading the data, make sure all data has been written to this directory by executing the following command as root:</para>
- <screen>
-<command>opcontrol --dump</command>
- </screen>
+ <screen>~]# <command>opcontrol --dump</command></screen>
<para>Each sample file name is based on the name of the executable. For example, the samples for the default event on a Pentium III processor for <command>/bin/bash</command> becomes:</para>
<screen>\{root\}/bin/bash/\{dep\}/\{root\}/bin/bash/CPU_CLK_UNHALTED.100000</screen>
<para>The following tools are available to profile the sample data once it has been collected:</para>
@@ -724,11 +804,11 @@
</itemizedlist>
<para>Use these tools, along with the binaries profiled, to generate reports that can be further analyzed.</para>
<warning>
- <title>Warning</title>
- <para>The executable being profiled must be used with these tools to analyze the data. If it must change after the data is collected, backup the executable used to create the samples as well as the sample files. Please note that the sample file and the binary have to agree. Making a backup isn't going to work if they do not match. <command>oparchive</command> can be used to address this problem.</para>
+ <title>Back up the executable and the sample files</title>
+ <para>The executable being profiled must be used with these tools to analyze the data. If it must change after the data is collected, back up the executable used to create the samples as well as the sample files. Please note that the sample file and the binary have to agree. Making a backup is not going to work if they do not match. <command>oparchive</command> can be used to address this problem.</para>
</warning>
- <para>Samples for each executable are written to a single sample file. Samples from each dynamically linked library are also written to a single sample file. While OProfile is running, if the executable being monitored changes and a sample file for the executable exists, the existing sample file is automatically deleted. Thus, if the existing sample file is needed, it must be backed up, along with the executable used to create it before replacing the executable with a new version. The oprofile analysis tools use the executable file that created the samples during analysis. If the executable changes the analysis tools will be unable to analyze the associated samples. Refer to <xref
- linkend="s1-oprofile-saving-data"/> for details on how to backup the sample file.</para>
+ <para>Samples for each executable are written to a single sample file. Samples from each dynamically linked library are also written to a single sample file. While OProfile is running, if the executable being monitored changes and a sample file for the executable exists, the existing sample file is automatically deleted. Thus, if the existing sample file is needed, it must be backed up, along with the executable used to create it before replacing the executable with a new version. The OProfile analysis tools use the executable file that created the samples during analysis. If the executable changes the analysis tools will be unable to analyze the associated samples. Refer to <xref
+ linkend="s1-oprofile-saving-data"/> for details on how to back up the sample file.</para>
<section
id="s2-oprofile-reading-opreport">
<title>Using <command>opreport</command>
@@ -749,8 +829,7 @@
</indexterm>
<para>The <command>opreport</command> tool provides an overview of all the executables being profiled.</para>
<para>The following is part of a sample output:</para>
- <screen>
-Profiling through timer interrupt
+ <screen>Profiling through timer interrupt
TIMER:0|
samples| %|
------------------
@@ -775,14 +854,13 @@ samples| %|
1 0.0038 libtermcap.so.2.0.8
1 0.0038 libX11.so.6.2
1 0.0038 libgthread-2.0.so.0.400.7
-1 0.0038 libwnck-1.so.4.9.0
-</screen>
+1 0.0038 libwnck-1.so.4.9.0</screen>
<para>Each executable is listed on its own line. The first column is the number of samples recorded for the executable. The second column is the percentage of samples relative to the total number of samples. The third column is the name of the executable.</para>
<para>Refer to the <command>opreport</command> man page for a list of available command line options, such as the <option>-r</option> option used to sort the output from the executable with the smallest number of samples to the one with the largest number of samples.</para>
</section>
<section
id="s2-oprofile-reading-opreport-single">
- <title>Using <command>opreport</command> on a Single Executable</title>
+ <title>Using opreport on a Single Executable</title>
<indexterm
significance="normal">
<primary>OProfile</primary>
@@ -799,33 +877,56 @@ samples| %|
<see>OProfile</see>
</indexterm>
<para>To retrieve more detailed profiled information about a specific executable, use <command>opreport</command>:</para>
- <screen>
-<command>opreport <replaceable><mode></replaceable>
- <replaceable><executable></replaceable>
- </command>
- </screen>
+ <screen>~]# <command>opreport <replaceable>mode</replaceable>
+ <replaceable>executable</replaceable></command></screen>
<para>
- <replaceable><executable></replaceable> must be the full path to the executable to be analyzed. <replaceable><mode></replaceable> must be one of the following:</para>
+ <replaceable>executable</replaceable> must be the full path to the executable to be analyzed. <replaceable>mode</replaceable> must be one of the following:</para>
<variablelist>
<varlistentry>
<term>
<option>-l</option>
</term>
<listitem>
- <para>List sample data by symbols. For example, the following is part of the output from running the command <command>opreport -l /lib/tls/libc-<replaceable><version></replaceable>.so</command>:</para>
- <screen> samples % symbol name 12 21.4286 __gconv_transform_utf8_internal 5 8.9286 _int_malloc 4 7.1429 malloc 3 5.3571 __i686.get_pc_thunk.bx 3 5.3571 _dl_mcount_wrapper_check 3 5.3571 mbrtowc 3 5.3571 memcpy 2 3.5714 _int_realloc 2 3.5714 _nl_intern_locale_data 2 3.5714 free 2 3.5714 strcmp 1 1.7857 __ctype_get_mb_cur_max 1 1.7857 __unregister_atfork 1 1.7857 __write_nocancel 1 1.7857 _dl_addr 1 1.7857 _int_free 1 1.7857 _itoa_word 1 1.7857 calc_eclosure_iter 1 1.7857 fopen@@GLIBC_2.1 1 1.7857 getpid 1 1.7857 memmove 1 1.7857 msort_with_tmp 1 1.7857 strcpy 1 1.7857 strlen 1 1.7857 vfprintf 1 1.7857 write </screen>
+ <para>List sample data by symbols. For example, the following is part of the output from running the command <command>opreport -l /lib/tls/libc-<replaceable>version</replaceable>.so</command>:</para>
+ <screen>samples % symbol name
+12 21.4286 __gconv_transform_utf8_internal
+5 8.9286 _int_malloc 4 7.1429 malloc
+3 5.3571 __i686.get_pc_thunk.bx
+3 5.3571 _dl_mcount_wrapper_check
+3 5.3571 mbrtowc
+3 5.3571 memcpy
+2 3.5714 _int_realloc
+2 3.5714 _nl_intern_locale_data
+2 3.5714 free
+2 3.5714 strcmp
+1 1.7857 __ctype_get_mb_cur_max
+1 1.7857 __unregister_atfork
+1 1.7857 __write_nocancel
+1 1.7857 _dl_addr
+1 1.7857 _int_free
+1 1.7857 _itoa_word
+1 1.7857 calc_eclosure_iter
+1 1.7857 fopen@@GLIBC_2.1
+1 1.7857 getpid
+1 1.7857 memmove
+1 1.7857 msort_with_tmp
+1 1.7857 strcpy
+1 1.7857 strlen
+1 1.7857 vfprintf
+1 1.7857 write</screen>
<para>The first column is the number of samples for the symbol, the second column is the percentage of samples for this symbol relative to the overall samples for the executable, and the third column is the symbol name.</para>
<para>To sort the output from the largest number of samples to the smallest (reverse order), use <option>-r</option> in conjunction with the <option>-l</option> option.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
- <option>-i <replaceable><symbol-name></replaceable>
+ <option>-i <replaceable>symbol-name</replaceable>
</option>
</term>
<listitem>
- <para>List sample data specific to a symbol name. For example, the following output is from the command <command>opreport -l -i __gconv_transform_utf8_internal /lib/tls/libc-<replaceable><version></replaceable>.so</command>:</para>
- <screen> samples % symbol name 12 100.000 __gconv_transform_utf8_internal </screen>
+ <para>List sample data specific to a symbol name. For example, the following output is from the command <command>opreport -l -i __gconv_transform_utf8_internal /lib/tls/libc-<replaceable>version</replaceable>.so</command>:</para>
+ <screen>samples % symbol name
+12 100.000 __gconv_transform_utf8_internal</screen>
<para>The first line is a summary for the symbol/executable combination.</para>
<para>The first column is the number of samples for the memory symbol. The second column is the percentage of samples for the memory address relative to the total number of samples for the symbol. The third column is the symbol name.</para>
</listitem>
@@ -835,15 +936,27 @@ samples| %|
<option>-d</option>
</term>
<listitem>
- <para>List sample data by symbols with more detail than <option>-l</option>. For example, the following output is from the command <command>opreport -l -d __gconv_transform_utf8_internal /lib/tls/libc-<replaceable><version></replaceable>.so</command>:</para>
- <screen> vma samples % symbol name 00a98640 12 100.000 __gconv_transform_utf8_internal 00a98640 1 8.3333 00a9868c 2 16.6667 00a9869a 1 8.3333 00a986c1 1 8.3333 00a98720 1 8.3333 00a98749 1 8.3333 00a98753 1 8.3333 00a98789 1 8.3333 00a98864 1 8.3333 00a98869 1 8.3333 00a98b08 1 8.3333 </screen>
+ <para>List sample data by symbols with more detail than <option>-l</option>. For example, the following output is from the command <command>opreport -l -d __gconv_transform_utf8_internal /lib/tls/libc-<replaceable>version</replaceable>.so</command>:</para>
+ <screen>vma samples % symbol name
+00a98640 12 100.000 __gconv_transform_utf8_internal
+00a98640 1 8.3333
+00a9868c 2 16.6667
+00a9869a 1 8.3333
+00a986c1 1 8.3333
+00a98720 1 8.3333
+00a98749 1 8.3333
+00a98753 1 8.3333
+00a98789 1 8.3333
+00a98864 1 8.3333
+00a98869 1 8.3333
+00a98b08 1 8.3333</screen>
<para>The data is the same as the <option>-l</option> option except that for each symbol, each virtual memory address used is shown. For each virtual memory address, the number of samples and percentage of samples relative to the number of samples for the symbol is displayed.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<option>-x</option>
- <replaceable><symbol-name></replaceable>
+ <replaceable>symbol-name</replaceable>
</term>
<listitem>
<para>Exclude the comma-separated list of symbols from the output.</para>
@@ -851,7 +964,7 @@ samples| %|
</varlistentry>
<varlistentry>
<term>
- <option>session</option>:<replaceable><name></replaceable>
+ <option>session</option>:<replaceable>name</replaceable>
</term>
<listitem>
<para>Specify the full path to the session or a directory relative to the <filename>/var/lib/oprofile/samples/</filename> directory.</para>
@@ -874,43 +987,31 @@ samples| %|
<primary/>
<see>OProfile</see>
</indexterm>
- <para>OProfile collects data on a system-wide basis for kernel- and user-space code running on the machine. However, once a module is loaded into the kernel, the information about the origin of the kernel module is lost. The module could have come from the initrd file on boot up, the directory with the various kernel modules, or a locally created kernel module. As a result when OProfile records sample for a module, it just lists the samples for the modules for an executable in the root directory, but this is unlikely to be the place with the actual code for the module. You will need to take some steps to make sure that analysis tools get the executable.</para>
- <para>For example on an AMD64 machine the sampling is set up to record "Data cache accesses" and "Data cache misses" and assuming you would like to see the data for the ext3 module:</para>
- <screen>
-<command>$ opreport /ext3</command>
-CPU: AMD64 processors, speed 797.948 MHz (estimated)
-Counted DATA_CACHE_ACCESSES events (Data cache accesses) with a unit mask of 0x00 (No unit mask) count 500000
-Counted DATA_CACHE_MISSES events (Data cache misses) with a unit mask of 0x00 (No unit mask) count 500000
-DATA_CACHE_ACC...|DATA_CACHE_MIS...|
-samples| %| samples| %|
-------------------------------------
-148721 100.000 1493 100.000 ext3
-</screen>
- <para>To get a more detailed view of the actions of the module, you will need to either have the module unstripped (e.g. installed from a custom build) or have the debuginfo RPM installed for the kernel.</para>
- <para>Find out which kernel is running, "uname -a", get the appropriate debuginfo rpm, and install on the machine.</para>
- <para>Then make a symbolic link so oprofile finds the code for the module in the correct place:</para>
- <screen>
- <command># ln -s /lib/modules/`uname -r`/kernel/fs/ext3/ext3.ko /ext3</command>
- </screen>
- <para>Then the detailed information can be obtained with:</para>
- <screen>
-<command># opreport image:/ext3 -l|more</command>
-warning: could not check that the binary file /ext3 has not been modified since the profile was taken. Results may be inaccurate.
-CPU: AMD64 processors, speed 797.948 MHz (estimated)
-Counted DATA_CACHE_ACCESSES events (Data cache accesses) with a unit mask of 0x00 (No unit mask) count 500000
-Counted DATA_CACHE_MISSES events (Data cache misses) with a unit mask of 0x00 (No unit mask) count 500000
-samples % samples % symbol name
-16728 11.2479 7 0.4689 ext3_group_sparse
-16454 11.0637 4 0.2679 ext3_count_free_blocks
-14583 9.8056 51 3.4159 ext3_fill_super
-8281 5.5681 129 8.6403 ext3_ioctl
-7810 5.2514 62 4.1527 ext3_write_info
-7286 4.8991 67 4.4876 ext3_ordered_writepage
-6509 4.3767 130 8.7073 ext3_new_inode
-6378 4.2886 156 10.4488 ext3_new_block
-5932 3.9887 87 5.8272 ext3_xattr_block_list
-...
-</screen>
+ <para>OProfile collects data on a system-wide basis for kernel- and user-space code running on the machine. However, once a module is loaded into the kernel, the information about the origin of the kernel module is lost. The module could have come from the <filename>initrd</filename> file on boot up, the directory with the various kernel modules, or a locally created kernel module. As a result, when OProfile records sample for a module, it just lists the samples for the modules for an executable in the root directory, but this is unlikely to be the place with the actual code for the module. You will need to take some steps to make sure that analysis tools get the executable.</para>
+ <para>To get a more detailed view of the actions of the module, you will need to either have the module "unstripped" (that is installed from a custom build) or have the <package>debuginfo</package> package installed for the kernel.</para>
+ <para>Find out which kernel is running with the <command>uname -a</command> command, obtain the appropriate <package>debuginfo</package> package and install it on the machine.</para>
+ <para>Then proceed with clearing out the samples from previous runs with the following command:</para>
+ <screen>~]# <command>opcontrol --reset</command></screen>
+ <para>To start the monitoring process, for example, on a machine with Westmere processor, run the following command:</para>
+ <screen>~]# <command>opcontrol --setup --vmlinux=/usr/lib/debug/lib/modules/`uname -r`/vmlinux \</command>
+<command>--event=CPU_CLK_UNHALTED:500000</command></screen>
+ <para>Then the detailed information, for instance, for the ext4 module can be obtained with:</para>
+ <screen>~]# <command>opreport /ext4 -l --image-path /lib/modules/`uname -r`/kernel</command>
+CPU: Intel Westmere microarchitecture, speed 2.667e+06 MHz (estimated)
+Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit mask of 0x00 (No unit mask) count 500000
+warning: could not check that the binary file /lib/modules/2.6.32-191.el6.x86_64/kernel/fs/ext4/ext4.ko has not been modified since the profile was taken. Results may be inaccurate.
+samples % symbol name
+1622 9.8381 ext4_iget
+1591 9.6500 ext4_find_entry
+1231 7.4665 __ext4_get_inode_loc
+783 4.7492 ext4_ext_get_blocks
+752 4.5612 ext4_check_dir_entry
+644 3.9061 ext4_mark_iloc_dirty
+583 3.5361 ext4_get_blocks
+583 3.5361 ext4_xattr_get
+479 2.9053 ext4_htree_store_dirent
+469 2.8447 ext4_get_group_desc
+414 2.5111 ext4_dx_find_entry</screen>
</section>
<section
id="s2-oprofile-reading-opannotate">
@@ -931,12 +1032,9 @@ samples % samples % symbol name
<see>OProfile</see>
</indexterm>
<para>The <command>opannotate</command> tool tries to match the samples for particular instructions to the corresponding lines in the source code. The resulting files generated should have the samples for the lines at the left. It also puts in a comment at the beginning of each function listing the total samples for the function.</para>
- <para>For this utility to work, the executable must be compiled with GCC's <option>-g</option> option. By default, &MAJOROS; packages are not compiled with this option.</para>
+ <para>For this utility to work, the appropriate <package>debuginfo</package> package for the executable must be installed on the system. By default, &MAJOROS; <package>debuginfo</package> packages are not installed together with their corresponding packages, which contain the executable, so that you have to obtain and install the <package>debuginfo</package> packages separately.</para>
<para>The general syntax for <command>opannotate</command> is as follows:</para>
- <screen>
-<command>opannotate --search-dirs <replaceable><src-dir></replaceable> --source <replaceable><executable></replaceable>
- </command>
- </screen>
+ <screen>~]# <command>opannotate --search-dirs <replaceable>src-dir</replaceable> --source <replaceable>executable</replaceable></command></screen>
<para>The directory containing the source code and the executable to be analyzed must be specified. Refer to the <command>opannotate</command> man page for a list of additional command line options.</para>
</section>
</section>
@@ -958,9 +1056,7 @@ samples % samples % symbol name
</primary>
</indexterm>
<para>The <filename>/dev/oprofile/</filename> directory contains the file system for OProfile. Use the <command>cat</command> command to display the values of the virtual files in this file system. For example, the following command displays the type of processor OProfile detected:</para>
- <screen>
-<command>cat /dev/oprofile/cpu_type</command>
- </screen>
+ <screen>~]# <command>cat /dev/oprofile/cpu_type</command></screen>
<para>A directory exists in <filename>/dev/oprofile/</filename> for each counter. For example, if there are 2 counters, the directories <filename>/dev/oprofile/0/</filename> and <filename>dev/oprofile/1/</filename> exist.</para>
<para>Each directory for a counter contains the following files:</para>
<itemizedlist>
@@ -978,6 +1074,10 @@ samples % samples % symbol name
</listitem>
<listitem>
<para>
+ <filename>extra</filename> — Used on machines with Nehalem processors to further specify the event to monitor.</para>
+ </listitem>
+ <listitem>
+ <para>
<filename>kernel</filename> — If 0, samples are not collected for this counter event when the processor is in kernel-space; if 1, samples are collected even if the processor is in kernel-space.</para>
</listitem>
<listitem>
@@ -990,9 +1090,7 @@ samples % samples % symbol name
</listitem>
</itemizedlist>
<para>The values of these files can be retrieved with the <command>cat</command> command. For example:</para>
- <screen>
-<command>cat /dev/oprofile/0/count</command>
- </screen>
+ <screen>~]# <command>cat /dev/oprofile/0/count</command></screen>
</section>
<section
id="s1-oprofile-example-usage">
@@ -1010,6 +1108,28 @@ samples % samples % symbol name
</itemizedlist>
</section>
<section
+ id="s1-oprofile-java-support">
+ <title>OProfile Support for Java</title>
+ <indexterm
+ significance="normal">
+ <primary>OProfile</primary>
+ <secondary>Java</secondary>
+ </indexterm>
+ <para>OProfile allows you to profile dynamically compiled code (also known as "just-in-time" or JIT code) of the Java Virtual Machine (JVM). OProfile in &MAJOROSVER; includes build-in support for the JVM Tools Interface (JVMTI) agent library, which supports Java 1.5 and higher.</para>
+ <section
+ id="s1-oprofile-java-profiling">
+ <title>Profiling Java Code</title>
+ <para>To profile JIT code from the Java Virtual Machine with the JVMTI agent, add the following to the JVM startup parameters:</para>
+ <screen><option>-agentlib:jvmti_oprofile</option></screen>
+ <note>
+ <title>Install the oprofile-jit package</title>
+ <para>The <package>oprofile-jit</package> package must be installed on the system in order to profile JIT code with OProfile.</para>
+ </note>
+ <para>To learn more about Java support in OProfile, refer to the OProfile Manual, which is linked from <xref
+ linkend="s1-oprofile-additional-resources"/>.</para>
+ </section>
+ </section>
+ <section
id="s1-oprofile-gui">
<title>Graphical Interface</title>
<indexterm
@@ -1030,7 +1150,8 @@ samples % samples % symbol name
<imageobject>
<imagedata
fileref="images/oprof-start-setup.png"
- format="PNG"/>
+ format="PNG"
+ scalefit="0"/>
</imageobject>
<textobject>
<para>
@@ -1056,7 +1177,7 @@ samples % samples % symbol name
<imagedata
fileref="images/oprof-start-config.png"
format="PNG"
- scalefit="1"/>
+ scalefit="0"/>
</imageobject>
<textobject>
<para>OProfile Configuration</para>
@@ -1064,11 +1185,25 @@ samples % samples % symbol name
</mediaobject>
</figure>
<para>If the <guilabel>Verbose</guilabel> option is selected, the <command>oprofiled</command> daemon log includes more information.</para>
- <para>If <guilabel>Per-application kernel samples files</guilabel> is selected, OProfile generates per-application profiles for the kernel and kernel modules as discussed in <xref
- linkend="s2-oprofile-starting-separate"/>. This is equivalent to the <command>opcontrol --separate=kernel</command> command. If <guilabel>Per-application shared libs samples files</guilabel> is selected, OProfile generates per-application profiles for libraries. This is equivalent to the <command>opcontrol --separate=library</command> command.</para>
+ <para>If <guilabel>Per-application profiles</guilabel> is selected, OProfile generates per-application profiles for libraries. This is equivalent to the <command>opcontrol --separate=library</command> command. If <guilabel>Per-application profiles, including kernel</guilabel> is selected, OProfile generates per-application profiles for the kernel and kernel modules as discussed in <xref
+ linkend="s2-oprofile-starting-separate"/>. This is equivalent to the <command>opcontrol --separate=kernel</command> command.</para>
<para>To force data to be written to samples files as discussed in <xref
- linkend="s1-oprofile-analyzing-data"/>, click the <guibutton>Flush profiler data</guibutton> button. This is equivalent to the <command>opcontrol --dump</command> command.</para>
- <para>To start OProfile from the graphical interface, click <guibutton>Start profiler</guibutton>. To stop the profiler, click <guibutton>Stop profiler</guibutton>. Exiting the application does not stop OProfile from sampling.</para>
+ linkend="s1-oprofile-analyzing-data"/>, click the <guibutton>Flush</guibutton> button. This is equivalent to the <command>opcontrol --dump</command> command.</para>
+ <para>To start OProfile from the graphical interface, click <guibutton>Start</guibutton>. To stop the profiler, click <guibutton>Stop</guibutton>. Exiting the application does not stop OProfile from sampling.</para>
+ </section>
+ <section
+ id="s1-oprofile-and-systemtap">
+ <title>OProfile and SystemTap</title>
+ <indexterm
+ significance="normal">
+ <primary>OProfile</primary>
+ <secondary>SystemTap</secondary>
+ </indexterm>
+ <para>SystemTap is a tracing and probing tool that allows users to study and monitor the activities of the operating system in fine detail. It provides information similar to the output of tools like <command>netstat</command>, <command>ps</command>, <command>top</command>, and <command>iostat</command>; however, SystemTap is designed to provide more filtering and analysis options for collected information.</para>
+ <para>While using OProfile is suggested in cases of collecting data on where and why the processor spends time in a particular area of code, it is less usable when finding out why the processor stays idle.</para>
+ <para>You might want to use SystemTap when instrumenting specific places in code. Because SystemTap allows you to run the code instrumentation without having to stop and restart the instrumentation, it is particularly useful for instrumenting the kernel and daemons.</para>
+ <para>For more information on SystemTap, refer to <xref
+ linkend="s2-oprofile-useful-websites"/> for the relevant SystemTap documentation.</para>
</section>
<section
id="s1-oprofile-additional-resources">
@@ -1085,7 +1220,7 @@ samples % samples % symbol name
<itemizedlist>
<listitem>
<para>
- <filename>/usr/share/doc/oprofile-<replaceable><version></replaceable>/oprofile.html</filename> — <citetitle>OProfile Manual</citetitle>
+ <filename>/usr/share/doc/oprofile-<replaceable>version</replaceable>/oprofile.html</filename> — <citetitle>OProfile Manual</citetitle>
</para>
</listitem>
<listitem>
@@ -1104,6 +1239,11 @@ samples % samples % symbol name
<ulink
url="http://oprofile.sourceforge.net/">http://oprofile.sourceforge.net/</ulink> — Contains the latest documentation, mailing lists, IRC channels, and more.</para>
</listitem>
+ <listitem>
+ <para>
+ <ulink
+ url="http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/SystemTap_Beginners_Guide/index.html">SystemTap Beginners Guide</ulink> — Provides basic instructions on how to use SystemTap to monitor different subsystems of &MAJOROS; in finer detail.</para>
+ </listitem>
</itemizedlist>
</section>
</section>
More information about the docs-commits
mailing list