Commit 7535249d authored by Len Brown's avatar Len Brown
Browse files

tools/power turbostat: update turbostat.8 



Update turbostat.8 to reflect new uncore frequency output (UncMHz)
Also, refresh examples.

Signed-off-by: default avatarLen Brown <len.brown@intel.com>
parent a5c6d65d

tools/power/x86/turbostat/turbostat.8

+124 −76
Original line number Diff line number Diff line
@@ -92,40 +92,66 @@ displays the statistics gathered since it was forked. 
.SH ROW DESCRIPTIONS

The system configuration dump (if --quiet is not used) is followed by statistics.  The first row of the statistics labels the content of each column (below).  The second row of statistics is the system summary line.  The system summary line has a '-' in the columns for the Package, Core, and CPU.  The contents of the system summary line depends on the type of column.  Columns that count items (eg. IRQ) show the sum across all CPUs in the system.  Columns that show a percentage show the average across all CPUs in the system.  Columns that dump raw MSR values simply show 0 in the summary.  After the system summary row, each row describes a specific Package/Core/CPU.  Note that if the --cpu parameter is used to limit which specific CPUs are displayed, turbostat will still collect statistics for all CPUs in the system and will still show the system summary for all CPUs in the system.

.SH COLUMN DESCRIPTIONS

.nf

.PP

\fBusec\fP For each CPU, the number of microseconds elapsed during counter collection, including thread migration -- if any.  This counter is disabled by default, and is enabled with "--enable usec", or --debug.  On the summary row, usec refers to the total elapsed time to collect the counters on all cpus.

.PP

\fBTime_Of_Day_Seconds\fP For each CPU, the gettimeofday(2) value (seconds.subsec since Epoch) when the counters ending the measurement interval were collected.  This column is disabled by default, and can be enabled with "--enable Time_Of_Day_Seconds" or "--debug".  On the summary row, Time_Of_Day_Seconds refers to the timestamp following collection of counters on the last CPU.

.PP

\fBCore\fP processor core number.  Note that multiple CPUs per core indicate support for Intel(R) Hyper-Threading Technology (HT).

.PP

\fBCPU\fP Linux CPU (logical processor) number.  Yes, it is okay that on many systems the CPUs are not listed in numerical order -- for efficiency reasons, turbostat runs in topology order, so HT siblings appear together.

.PP

\fBPackage\fP processor package number -- not present on systems with a single processor package.

.PP

\fBAvg_MHz\fP number of cycles executed divided by time elapsed.  Note that this includes idle-time when 0 instructions are executed.

.PP

\fBBusy%\fP percent of the measurement interval that the CPU executes instructions, aka. % of time in "C0" state.

.PP

\fBBzy_MHz\fP average clock rate while the CPU was not idle (ie. in "c0" state).

.PP

\fBTSC_MHz\fP average MHz that the TSC ran during the entire interval.

.PP

\fBIRQ\fP The number of interrupts serviced by that CPU during the measurement interval.  The system total line is the sum of interrupts serviced across all CPUs.  turbostat parses /proc/interrupts to generate this summary.

.PP

\fBSMI\fP The number of System Management Interrupts  serviced CPU during the measurement interval.  While this counter is actually per-CPU, SMI are triggered on all processors, so the number should be the same for all CPUs.

.PP

\fBC1, C2, C3...\fP The number times Linux requested the C1, C2, C3 idle state during the measurement interval.  The system summary line shows the sum for all CPUs.  These are C-state names as exported in /sys/devices/system/cpu/cpu*/cpuidle/state*/name.  While their names are generic, their attributes are processor specific. They the system description section of output shows what MWAIT sub-states they are mapped to on each system.

.PP

\fBC1%, C2%, C3%\fP The residency percentage that Linux requested C1, C2, C3....  The system summary is the average of all CPUs in the system.  Note that these are software, reflecting what was requested.  The hardware counters reflect what was actually achieved.

.PP

\fBCPU%c1, CPU%c3, CPU%c6, CPU%c7\fP show the percentage residency in hardware core idle states.  These numbers are from hardware residency counters.

.PP

\fBCoreTmp\fP Degrees Celsius reported by the per-core Digital Thermal Sensor.

.PP

\fBPkgTmp\fP Degrees Celsius reported by the per-package Package Thermal Monitor.

.PP

\fBGFX%rc6\fP The percentage of time the GPU is in the "render C6" state, rc6, during the measurement interval. From /sys/class/drm/card0/power/rc6_residency_ms.

.PP

\fBGFXMHz\fP Instantaneous snapshot of what sysfs presents at the end of the measurement interval. From /sys/class/graphics/fb0/device/drm/card0/gt_cur_freq_mhz.

.PP

\fBPkg%pc2, Pkg%pc3, Pkg%pc6, Pkg%pc7\fP percentage residency in hardware package idle states.  These numbers are from hardware residency counters.

.PP

\fBPkgWatt\fP Watts consumed by the whole package.

.PP

\fBCorWatt\fP Watts consumed by the core part of the package.

.PP

\fBGFXWatt\fP Watts consumed by the Graphics part of the package -- available only on client processors.

.PP

\fBRAMWatt\fP Watts consumed by the DRAM DIMMS -- available only on server processors.

.PP

\fBPKG_%\fP percent of the interval that RAPL throttling was active on the Package.  Note that the system summary is the sum of the package throttling time, and thus may be higher than 100% on a multi-package system.  Note that the meaning of this field is model specific.  For example, some hardware increments this counter when RAPL responds to thermal limits, but does not increment this counter when RAPL responds to power limits.  Comparing PkgWatt and PkgTmp to system limits is necessary.

.PP

\fBRAM_%\fP percent of the interval that RAPL throttling was active on DRAM.

.fi

.PP

\fBUncMHz\fP uncore MHz, instantaneous sample.

.SH TOO MUCH INFORMATION EXAMPLE

By default, turbostat dumps all possible information -- a system configuration header, followed by columns for all counters.

This is ideal for remote debugging, use the "--out" option to save everything to a text file, and get that file to the expert helping you debug.

.PP

When you are not interested in all that information, and there are several ways to see only what you want.  First the "--quiet" option will skip the configuration information, and turbostat will show only the counter columns.  Second, you can reduce the columns with the "--hide" and "--show" options.  If you use the "--show" option, then turbostat will show only the columns you list.  If you use the "--hide" option, turbostat will show all columns, except the ones you list.

.PP

To find out what columns are available for --show and --hide, the "--list" option is available.  For convenience, the special strings "sysfs" can be used to refer to all of the sysfs C-state counters at once:

To find out what columns are available for --show and --hide, the "--list" option is available.  Usually, the CATEGORY names above are used to refer to groups of counters.  Also, for convenience, the special string "sysfs" can be used to refer to all of the sysfs C-state counters at once:

.PP

.nf

sudo ./turbostat --show sysfs --quiet sleep 10

10.003837 sec
@@ -158,32 +184,29 @@ Without a command to fork, turbostat displays statistics ever 5 seconds. 
Periodic output goes to stdout, by default, unless --out is used to specify an output file.

The 5-second interval can be changed with the "-i sec" option.

.nf

sudo ./turbostat --quiet --hide sysfs,IRQ,SMI,CoreTmp,PkgTmp,GFX%rc6,GFXMHz,PkgWatt,CorWatt,GFXWatt

	Core	CPU	Avg_MHz	Busy%	Bzy_MHz	TSC_MHz	CPU%c1	CPU%c3	CPU%c6	CPU%c7

	-	-	488	12.52	3900	3498	12.50	0.00	0.00	74.98

	0	0	5	0.13	3900	3498	99.87	0.00	0.00	0.00

	0	4	3897	99.99	3900	3498	0.01

	1	1	0	0.00	3856	3498	0.01	0.00	0.00	99.98

	1	5	0	0.00	3861	3498	0.01

	2	2	1	0.02	3889	3498	0.03	0.00	0.00	99.95

	2	6	0	0.00	3863	3498	0.05

	3	3	0	0.01	3869	3498	0.02	0.00	0.00	99.97

	3	7	0	0.00	3878	3498	0.03

	Core	CPU	Avg_MHz	Busy%	Bzy_MHz	TSC_MHz	CPU%c1	CPU%c3	CPU%c6	CPU%c7

	-	-	491	12.59	3900	3498	12.42	0.00	0.00	74.99

	0	0	27	0.69	3900	3498	99.31	0.00	0.00	0.00

	0	4	3898	99.99	3900	3498	0.01

	1	1	0	0.00	3883	3498	0.01	0.00	0.00	99.99

	1	5	0	0.00	3898	3498	0.01

	2	2	0	0.01	3889	3498	0.02	0.00	0.00	99.98

	2	6	0	0.00	3889	3498	0.02

	3	3	0	0.00	3856	3498	0.01	0.00	0.00	99.99

	3	7	0	0.00	3897	3498	0.01

sudo turbostat --quiet --show CPU,frequency

	Core	CPU	Avg_MHz	Busy%	Bzy_MHz	TSC_MHz	CPU%c7	UncMhz

	-	-	524	12.48	4198	3096	74.53	3800

	0	0	4	0.09	4081	3096	98.88	3800

	0	4	1	0.02	4063	3096

	1	1	2	0.06	4063	3096	99.60

	1	5	2	0.05	4070	3096

	2	2	4178	99.52	4199	3096	0.00

	2	6	3	0.08	4159	3096

	3	3	1	0.04	4046	3096	99.66

	3	7	0	0.01	3989	3096

	Core	CPU	Avg_MHz	Busy%	Bzy_MHz	TSC_MHz	CPU%c7	UncMhz

	-	-	525	12.52	4198	3096	74.54	3800

	0	0	4	0.10	4051	3096	99.49	3800

	0	4	2	0.04	3993	3096

	1	1	3	0.07	4054	3096	99.56

	1	5	4	0.10	4018	3096

	2	2	4178	99.51	4199	3096	0.00

	2	6	4	0.09	4143	3096

	3	3	2	0.06	4026	3096	99.10

	3	7	7	0.17	4074	3096

.fi

This example also shows the use of the --hide option to skip columns that are not wanted.

Note that cpu4 in this example is 99.99% busy, while the other CPUs are all under 1% busy.

Notice that cpu4's HT sibling is cpu0, which is under 1% busy, but can get into CPU%c1 only,

because its cpu4's activity on shared hardware keeps it from entering a deeper C-state.

This example also shows the use of the --show option to show only the desired columns.



.SH SYSTEM CONFIGURATION INFORMATION EXAMPLE
@@ -191,61 +214,86 @@ By default, turbostat always dumps system configuration information 
before taking measurements.  In the example above, "--quiet" is used

to suppress that output.  Here is an example of the configuration information:

.nf

turbostat version 2017.02.15 - Len Brown <lenb@kernel.org>

CPUID(0): GenuineIntel 13 CPUID levels; family:model:stepping 0x6:3c:3 (6:60:3)

CPUID(1): SSE3 MONITOR - EIST TM2 TSC MSR ACPI-TM TM

CPUID(6): APERF, TURBO, DTS, PTM, No-HWP, No-HWPnotify, No-HWPwindow, No-HWPepp, No-HWPpkg, EPB

cpu4: MSR_IA32_MISC_ENABLE: 0x00850089 (TCC EIST No-MWAIT PREFETCH TURBO)

CPUID(7): No-SGX

cpu4: MSR_MISC_PWR_MGMT: 0x00400000 (ENable-EIST_Coordination DISable-EPB DISable-OOB)

RAPL: 3121 sec. Joule Counter Range, at 84 Watts

cpu4: MSR_PLATFORM_INFO: 0x80838f3012300

turbostat version 2022.04.16 - Len Brown <lenb@kernel.org>

Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.18.0-rc6-00001-ge6891250e3b5 ...

CPUID(0): GenuineIntel 0x16 CPUID levels

CPUID(1): family:model:stepping 0x6:9e:9 (6:158:9) microcode 0xea

CPUID(0x80000000): max_extended_levels: 0x80000008

CPUID(1): SSE3 MONITOR - EIST TM2 TSC MSR ACPI-TM HT TM

CPUID(6): APERF, TURBO, DTS, PTM, HWP, HWPnotify, HWPwindow, HWPepp, No-HWPpkg, EPB

cpu7: MSR_IA32_MISC_ENABLE: 0x00850089 (TCC EIST MWAIT PREFETCH TURBO)

CPUID(7): SGX

cpu7: MSR_IA32_FEATURE_CONTROL: 0x00000005 (Locked )

CPUID(0x15): eax_crystal: 2 ebx_tsc: 258 ecx_crystal_hz: 0

TSC: 3096 MHz (24000000 Hz * 258 / 2 / 1000000)

CPUID(0x16): base_mhz: 3100 max_mhz: 4200 bus_mhz: 100

cpu7: MSR_MISC_PWR_MGMT: 0x00401cc0 (ENable-EIST_Coordination DISable-EPB DISable-OOB)

RAPL: 5825 sec. Joule Counter Range, at 45 Watts

cpu7: MSR_PLATFORM_INFO: 0x80839f1011f00

8 * 100.0 = 800.0 MHz max efficiency frequency

35 * 100.0 = 3500.0 MHz base frequency

cpu4: MSR_IA32_POWER_CTL: 0x0004005d (C1E auto-promotion: DISabled)

cpu4: MSR_TURBO_RATIO_LIMIT: 0x25262727

37 * 100.0 = 3700.0 MHz max turbo 4 active cores

38 * 100.0 = 3800.0 MHz max turbo 3 active cores

39 * 100.0 = 3900.0 MHz max turbo 2 active cores

39 * 100.0 = 3900.0 MHz max turbo 1 active cores

cpu4: MSR_CONFIG_TDP_NOMINAL: 0x00000023 (base_ratio=35)

cpu4: MSR_CONFIG_TDP_LEVEL_1: 0x00000000 ()

cpu4: MSR_CONFIG_TDP_LEVEL_2: 0x00000000 ()

cpu4: MSR_CONFIG_TDP_CONTROL: 0x80000000 ( lock=1)

cpu4: MSR_TURBO_ACTIVATION_RATIO: 0x00000000 (MAX_NON_TURBO_RATIO=0 lock=0)

cpu4: MSR_PKG_CST_CONFIG_CONTROL: 0x1e000400 (UNdemote-C3, UNdemote-C1, demote-C3, demote-C1, UNlocked: pkg-cstate-limit=0: pc0)

cpu4: POLL: CPUIDLE CORE POLL IDLE

cpu4: C1: MWAIT 0x00

cpu4: C1E: MWAIT 0x01

cpu4: C3: MWAIT 0x10

cpu4: C6: MWAIT 0x20

cpu4: C7s: MWAIT 0x32

cpu4: MSR_MISC_FEATURE_CONTROL: 0x00000000 (L2-Prefetch L2-Prefetch-pair L1-Prefetch L1-IP-Prefetch)

cpu0: MSR_IA32_ENERGY_PERF_BIAS: 0x00000006 (balanced)

cpu0: MSR_CORE_PERF_LIMIT_REASONS, 0x31200000 (Active: ) (Logged: Transitions, MultiCoreTurbo, Amps, Auto-HWP, )

cpu0: MSR_GFX_PERF_LIMIT_REASONS, 0x00000000 (Active: ) (Logged: )

cpu0: MSR_RING_PERF_LIMIT_REASONS, 0x0d000000 (Active: ) (Logged: Amps, PkgPwrL1, PkgPwrL2, )

31 * 100.0 = 3100.0 MHz base frequency

cpu7: MSR_IA32_POWER_CTL: 0x002c005d (C1E auto-promotion: DISabled)

cpu7: MSR_TURBO_RATIO_LIMIT: 0x2728292a

39 * 100.0 = 3900.0 MHz max turbo 4 active cores

40 * 100.0 = 4000.0 MHz max turbo 3 active cores

41 * 100.0 = 4100.0 MHz max turbo 2 active cores

42 * 100.0 = 4200.0 MHz max turbo 1 active cores

cpu7: MSR_CONFIG_TDP_NOMINAL: 0x0000001f (base_ratio=31)

cpu7: MSR_CONFIG_TDP_LEVEL_1: 0x00000000 ()

cpu7: MSR_CONFIG_TDP_LEVEL_2: 0x00000000 ()

cpu7: MSR_CONFIG_TDP_CONTROL: 0x80000000 ( lock=1)

cpu7: MSR_TURBO_ACTIVATION_RATIO: 0x00000000 (MAX_NON_TURBO_RATIO=0 lock=0)

cpu7: MSR_PKG_CST_CONFIG_CONTROL: 0x1e008008 (UNdemote-C3, UNdemote-C1, demote-C3, demote-C1, locked, pkg-cstate-limit=8 (unlimited))

Uncore Frequency pkg0 die0: 800 - 3900 MHz (800 - 3900 MHz)

/dev/cpu_dma_latency: 2000000000 usec (default)

current_driver: intel_idle

current_governor: menu

current_governor_ro: menu

cpu7: POLL: CPUIDLE CORE POLL IDLE

cpu7: C1: MWAIT 0x00

cpu7: C1E: MWAIT 0x01

cpu7: C3: MWAIT 0x10

cpu7: C6: MWAIT 0x20

cpu7: C7s: MWAIT 0x33

cpu7: C8: MWAIT 0x40

cpu7: C9: MWAIT 0x50

cpu7: C10: MWAIT 0x60

cpu7: cpufreq driver: intel_pstate

cpu7: cpufreq governor: performance

cpufreq intel_pstate no_turbo: 0

cpu7: MSR_MISC_FEATURE_CONTROL: 0x00000000 (L2-Prefetch L2-Prefetch-pair L1-Prefetch L1-IP-Prefetch)

cpu0: MSR_PM_ENABLE: 0x00000001 (HWP)

cpu0: MSR_HWP_CAPABILITIES: 0x01101f53 (high 83 guar 31 eff 16 low 1)

cpu0: MSR_HWP_REQUEST: 0x00005353 (min 83 max 83 des 0 epp 0x0 window 0x0 pkg 0x0)

cpu0: MSR_HWP_INTERRUPT: 0x00000001 (EN_Guaranteed_Perf_Change, Dis_Excursion_Min)

cpu0: MSR_HWP_STATUS: 0x00000004 (No-Guaranteed_Perf_Change, No-Excursion_Min)

cpu0: EPB: 6 (balanced)

cpu0: MSR_RAPL_POWER_UNIT: 0x000a0e03 (0.125000 Watts, 0.000061 Joules, 0.000977 sec.)

cpu0: MSR_PKG_POWER_INFO: 0x000002a0 (84 W TDP, RAPL 0 - 0 W, 0.000000 sec.)

cpu0: MSR_PKG_POWER_LIMIT: 0x428348001a82a0 (UNlocked)

cpu0: PKG Limit #1: ENabled (84.000000 Watts, 8.000000 sec, clamp DISabled)

cpu0: PKG Limit #2: ENabled (105.000000 Watts, 0.002441* sec, clamp DISabled)

cpu0: MSR_PKG_POWER_INFO: 0x00000168 (45 W TDP, RAPL 0 - 0 W, 0.000000 sec.)

cpu0: MSR_PKG_POWER_LIMIT: 0x42820800218208 (UNlocked)

cpu0: PKG Limit #1: ENabled (65.000 Watts, 64.000000 sec, clamp ENabled)

cpu0: PKG Limit #2: ENabled (65.000 Watts, 0.002441* sec, clamp DISabled)

cpu0: MSR_VR_CURRENT_CONFIG: 0x00000000

cpu0: PKG Limit #4: 0.000000 Watts (UNlocked)

cpu0: MSR_DRAM_POWER_LIMIT: 0x5400de00000000 (UNlocked)

cpu0: DRAM Limit: DISabled (0.000 Watts, 0.000977 sec, clamp DISabled)

cpu0: MSR_PP0_POLICY: 0

cpu0: MSR_PP0_POWER_LIMIT: 0x00000000 (UNlocked)

cpu0: Cores Limit: DISabled (0.000000 Watts, 0.000977 sec, clamp DISabled)

cpu0: Cores Limit: DISabled (0.000 Watts, 0.000977 sec, clamp DISabled)

cpu0: MSR_PP1_POLICY: 0

cpu0: MSR_PP1_POWER_LIMIT: 0x00000000 (UNlocked)

cpu0: GFX Limit: DISabled (0.000000 Watts, 0.000977 sec, clamp DISabled)

cpu0: MSR_IA32_TEMPERATURE_TARGET: 0x00641400 (100 C)

cpu0: MSR_IA32_PACKAGE_THERM_STATUS: 0x884c0800 (24 C)

cpu0: MSR_IA32_THERM_STATUS: 0x884c0000 (24 C +/- 1)

cpu1: MSR_IA32_THERM_STATUS: 0x88510000 (19 C +/- 1)

cpu2: MSR_IA32_THERM_STATUS: 0x884e0000 (22 C +/- 1)

cpu3: MSR_IA32_THERM_STATUS: 0x88510000 (19 C +/- 1)

cpu4: MSR_PKGC3_IRTL: 0x00008842 (valid, 67584 ns)

cpu4: MSR_PKGC6_IRTL: 0x00008873 (valid, 117760 ns)

cpu4: MSR_PKGC7_IRTL: 0x00008891 (valid, 148480 ns)

cpu0: GFX Limit: DISabled (0.000 Watts, 0.000977 sec, clamp DISabled)

cpu0: MSR_IA32_TEMPERATURE_TARGET: 0x00640000 (100 C) (100 default - 0 offset)

cpu0: MSR_IA32_PACKAGE_THERM_STATUS: 0x88200800 (68 C)

cpu0: MSR_IA32_PACKAGE_THERM_INTERRUPT: 0x00000003 (100 C, 100 C)

cpu7: MSR_PKGC3_IRTL: 0x0000884e (valid, 79872 ns)

cpu7: MSR_PKGC6_IRTL: 0x00008876 (valid, 120832 ns)

cpu7: MSR_PKGC7_IRTL: 0x00008894 (valid, 151552 ns)

cpu7: MSR_PKGC8_IRTL: 0x000088fa (valid, 256000 ns)

cpu7: MSR_PKGC9_IRTL: 0x0000894c (valid, 339968 ns)

cpu7: MSR_PKGC10_IRTL: 0x00008bf2 (valid, 1034240 ns)

.fi

.PP

The \fBmax efficiency\fP frequency, a.k.a. Low Frequency Mode, is the frequency

available at the minimum package voltage.  The \fBTSC frequency\fP is the base

frequency of the processor -- this should match the brand string