|
HTML rendering created 2024-06-26 by Michael Kerrisk, author of The Linux Programming Interface. For details of in-depth Linux/UNIX system programming training courses that I teach, look here. Hosting by jambit GmbH. |
|
|
NAME | SYNOPSIS | DESCRIPTION | OPTIONS | PROBE SYNTAX | ESCAPED CHARACTER | PROBE ARGUMENT | TYPES | LINE SYNTAX | LAZY MATCHING | FILTER PATTERN | EXAMPLES | PERMISSIONS AND SYSCTL | SEE ALSO | COLOPHON |
|
|
|
PERF-PROBE(1) perf Manual PERF-PROBE(1)
perf-probe - Define new dynamic tracepoints
perf probe [options] --add=PROBE [...]
or
perf probe [options] PROBE
or
perf probe [options] --del=[GROUP:]EVENT [...]
or
perf probe --list[=[GROUP:]EVENT]
or
perf probe [options] --line=LINE
or
perf probe [options] --vars=PROBEPOINT
or
perf probe [options] --funcs
or
perf probe [options] --definition=PROBE [...]
This command defines dynamic tracepoint events, by symbol and
registers without debuginfo, or by C expressions (C line numbers,
C function names, and C local variables) with debuginfo.
-k, --vmlinux=PATH
Specify vmlinux path which has debuginfo (Dwarf binary). Only
when using this with --definition, you can give an offline
vmlinux file.
-m, --module=MODNAME|PATH
Specify module name in which perf-probe searches probe points
or lines. If a path of module file is passed, perf-probe
treat it as an offline module (this means you can add a probe
on a module which has not been loaded yet).
-s, --source=PATH
Specify path to kernel source.
-v, --verbose
Be more verbose (show parsed arguments, etc). Can not use
with -q.
-q, --quiet
Do not show any warnings or messages. Can not use with -v.
-a, --add=
Define a probe event (see PROBE SYNTAX for detail).
-d, --del=
Delete probe events. This accepts glob wildcards(*, ?) and
character classes(e.g. [a-z], [!A-Z]).
-l, --list[=[GROUP:]EVENT]
List up current probe events. This can also accept filtering
patterns of event names. When this is used with --cache, perf
shows all cached probes instead of the live probes.
-L, --line=
Show source code lines which can be probed. This needs an
argument which specifies a range of the source code. (see
LINE SYNTAX for detail)
-V, --vars=
Show available local variables at given probe point. The
argument syntax is same as PROBE SYNTAX, but NO ARGs.
--externs
(Only for --vars) Show external defined variables in addition
to local variables.
--no-inlines
(Only for --add) Search only for non-inlined functions. The
functions which do not have instances are ignored.
-F, --funcs[=FILTER]
Show available functions in given module or kernel. With
-x/--exec, can also list functions in a user space executable
/ shared library. This also can accept a FILTER rule
argument.
-D, --definition=
Show trace-event definition converted from given probe-event
instead of write it into tracing/[k,u]probe_events.
--filter=FILTER
(Only for --vars and --funcs) Set filter. FILTER is a
combination of glob pattern, see FILTER PATTERN for detail.
Default FILTER is "!k???tab_* & !crc_*" for --vars, and "!_*"
for --funcs. If several filters are specified, only the last
filter is used.
-f, --force
Forcibly add events with existing name.
-n, --dry-run
Dry run. With this option, --add and --del doesn’t execute
actual adding and removal operations.
--cache
(With --add) Cache the probes. Any events which successfully
added are also stored in the cache file. (With --list) Show
cached probes. (With --del) Remove cached probes.
--max-probes=NUM
Set the maximum number of probe points for an event. Default
is 128.
--target-ns=PID: Obtain mount namespace information from the
target pid. This is used when creating a uprobe for a process
that resides in a different mount namespace from the perf(1)
utility.
-x, --exec=PATH
Specify path to the executable or shared library file for
user space tracing. Can also be used with --funcs option.
--demangle
Demangle application symbols. --no-demangle is also available
for disabling demangling.
--demangle-kernel
Demangle kernel symbols. --no-demangle-kernel is also
available for disabling kernel demangling.
In absence of -m/-x options, perf probe checks if the first
argument after the options is an absolute path name. If its an
absolute path, perf probe uses it as a target module/target user
space binary to probe.
Probe points are defined by following syntax.
1) Define event based on function name
[[GROUP:]EVENT=]FUNC[@SRC][:RLN|+OFFS|%return|;PTN] [ARG ...]
2) Define event based on source file with line number
[[GROUP:]EVENT=]SRC:ALN [ARG ...]
3) Define event based on source file with lazy pattern
[[GROUP:]EVENT=]SRC;PTN [ARG ...]
4) Pre-defined SDT events or cached event with name
%[sdt_PROVIDER:]SDTEVENT
or,
sdt_PROVIDER:SDTEVENT
EVENT specifies the name of new event, if omitted, it will be set
the name of the probed function, and for return probes, a
"__return" suffix is automatically added to the function name.
You can also specify a group name by GROUP, if omitted, set probe
is used for kprobe and probe_<bin> is used for uprobe. Note that
using existing group name can conflict with other events.
Especially, using the group name reserved for kernel modules can
hide embedded events in the modules. FUNC specifies a probed
function name, and it may have one of the following options;
+OFFS is the offset from function entry address in bytes, :RLN is
the relative-line number from function entry line, and %return
means that it probes function return. And ;PTN means lazy
matching pattern (see LAZY MATCHING). Note that ;PTN must be the
end of the probe point definition. In addition, @SRC specifies a
source file which has that function. It is also possible to
specify a probe point by the source line number or lazy matching
by using SRC:ALN or SRC;PTN syntax, where SRC is the source file
path, :ALN is the line number and ;PTN is the lazy matching
pattern. ARG specifies the arguments of this probe point, (see
PROBE ARGUMENT). SDTEVENT and PROVIDER is the pre-defined event
name which is defined by user SDT (Statically Defined Tracing) or
the pre-cached probes with event name. Note that before using the
SDT event, the target binary (on which SDT events are defined)
must be scanned by perf-buildid-cache(1) to make SDT events as
cached events.
For details of the SDT, see below.
https://sourceware.org/gdb/onlinedocs/gdb/Static-Probe-Points.html
In the probe syntax, =, @, +, : and ; are treated as a special
character. You can use a backslash (\) to escape the special
characters. This is useful if you need to probe on a specific
versioned symbols, like @GLIBC_... suffixes, or also you need to
specify a source file which includes the special characters. Note
that usually single backslash is consumed by shell, so you might
need to pass double backslash (\\) or wrapping with single quotes
('AAA\@BBB'). See EXAMPLES how it is used.
Each probe argument follows below syntax.
[NAME=]LOCALVAR|$retval|%REG|@SYMBOL[:TYPE][@user]
NAME specifies the name of this argument (optional). You can use
the name of local variable, local data structure member (e.g.
var→field, var.field2), local array with fixed index (e.g.
array[1], var→array[0], var→pointer[2]), or kprobe-tracer
argument format (e.g. $retval, %ax, etc). Note that the name of
this argument will be set as the last member name if you specify
a local data structure member (e.g. field2 for
var→field1.field2.) $vars and $params special arguments are also
available for NAME, $vars is expanded to the local variables
(including function parameters) which can access at given probe
point. $params is expanded to only the function parameters. TYPE
casts the type of this argument (optional). If omitted, perf
probe automatically set the type based on debuginfo (*).
Currently, basic types (u8/u16/u32/u64/s8/s16/s32/s64),
hexadecimal integers (x/x8/x16/x32/x64), signedness casting
(u/s), "string" and bitfield are supported. (see TYPES for
detail) On x86 systems %REG is always the short form of the
register: for example %AX. %RAX or %EAX is not valid. "@user" is
a special attribute which means the LOCALVAR will be treated as a
user-space memory. This is only valid for kprobe event.
Basic types (u8/u16/u32/u64/s8/s16/s32/s64) and hexadecimal
integers (x8/x16/x32/x64) are integer types. Prefix s and u means
those types are signed and unsigned respectively, and x means
that is shown in hexadecimal format. Traced arguments are shown
in decimal (sNN/uNN) or hex (xNN). You can also use s or u to
specify only signedness and leave its size auto-detected by perf
probe. Moreover, you can use x to explicitly specify to be shown
in hexadecimal (the size is also auto-detected). String type is a
special type, which fetches a "null-terminated" string from
kernel space. This means it will fail and store NULL if the
string container has been paged out. You can specify string type
only for the local variable or structure member which is an array
of or a pointer to char or unsigned char type. Bitfield is
another special type, which takes 3 parameters, bit-width,
bit-offset, and container-size (usually 32). The syntax is;
b<bit-width>@<bit-offset>/<container-size>
Line range is described by following syntax.
"FUNC[@SRC][:RLN[+NUM|-RLN2]]|SRC[:ALN[+NUM|-ALN2]]"
FUNC specifies the function name of showing lines. RLN is the
start line number from function entry line, and RLN2 is the end
line number. As same as probe syntax, SRC means the source file
path, ALN is start line number, and ALN2 is end line number in
the file. It is also possible to specify how many lines to show
by using NUM. Moreover, FUNC@SRC combination is good for
searching a specific function when several functions share same
name. So, "source.c:100-120" shows lines between 100th to 120th
in source.c file. And "func:10+20" shows 20 lines from 10th line
of func function.
The lazy line matching is similar to glob matching but ignoring
spaces in both of pattern and target. So this accepts
wildcards(*, ?) and character classes(e.g. [a-z], [!A-Z]).
e.g. a=* can matches a=b, a = b, a == b and so on.
This provides some sort of flexibility and robustness to probe
point definitions against minor code changes. For example, actual
10th line of schedule() can be moved easily by modifying
schedule(), but the same line matching rq=cpu_rq* may still exist
in the function.)
The filter pattern is a glob matching pattern(s) to filter
variables. In addition, you can use "!" for specifying filter-out
rule. You also can give several rules combined with "&" or "|",
and fold those rules as one rule by using "(" ")".
e.g. With --filter "foo* | bar*", perf probe -V shows variables
which start with "foo" or "bar". With --filter "!foo* & *bar",
perf probe -V shows variables which don’t start with "foo" and
end with "bar", like "fizzbar". But "foobar" is filtered out.
Display which lines in schedule() can be probed:
./perf probe --line schedule
Add a probe on schedule() function 12th line with recording cpu
local variable:
./perf probe schedule:12 cpu
or
./perf probe --add='schedule:12 cpu'
Add one or more probes which has the name start with "schedule".
./perf probe schedule*
or
./perf probe --add='schedule*'
Add probes on lines in schedule() function which calls
update_rq_clock().
./perf probe 'schedule;update_rq_clock*'
or
./perf probe --add='schedule;update_rq_clock*'
Delete all probes on schedule().
./perf probe --del='schedule*'
Add probes at zfree() function on /bin/zsh
./perf probe -x /bin/zsh zfree or ./perf probe /bin/zsh zfree
Add probes at malloc() function on libc
./perf probe -x /lib/libc.so.6 malloc or ./perf probe /lib/libc.so.6 malloc
Add a uprobe to a target process running in a different mount
namespace
./perf probe --target-ns <target pid> -x /lib64/libc.so.6 malloc
Add a USDT probe to a target process running in a different mount
namespace
./perf probe --target-ns <target pid> -x /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.121-0.b13.el7_3.x86_64/jre/lib/amd64/server/libjvm.so %sdt_hotspot:thread__sleep__end
Add a probe on specific versioned symbol by backslash escape
./perf probe -x /lib64/libc-2.25.so 'malloc_get_state\@GLIBC_2.2.5'
Add a probe in a source file using special characters by
backslash escape
./perf probe -x /opt/test/a.out 'foo\+bar.c:4'
Since perf probe depends on ftrace (tracefs) and kallsyms
(/proc/kallsyms), you have to care about the permission and some
sysctl knobs.
• Since tracefs and kallsyms requires root or privileged user
to access it, the following perf probe commands also require
it; --add, --del, --list (except for --cache option)
• The system admin can remount the tracefs with 755 (sudo mount
-o remount,mode=755 /sys/kernel/tracing/) to allow
unprivileged user to run the perf probe --list command.
• /proc/sys/kernel/kptr_restrict = 2 (restrict all users) also
prevents perf probe to retrieve the important information
from kallsyms. You also need to set to 1 (restrict non
CAP_SYSLOG users) for the above commands. Since the
user-space probe doesn’t need to access kallsyms, this is
only for probing the kernel function (kprobes).
• Since the perf probe commands read the vmlinux (for kernel)
and/or the debuginfo file (including user-space application),
you need to ensure that you can read those files.
perf-trace(1), perf-record(1), perf-buildid-cache(1)
This page is part of the perf (Performance analysis tools for
Linux (in Linux source tree)) project. Information about the
project can be found at
⟨https://perf.wiki.kernel.org/index.php/Main_Page⟩. If you have a
bug report for this manual page, send it to
[email protected]. This page was obtained from the
project's upstream Git repository
⟨http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git⟩
on 2024-06-14. (At that time, the date of the most recent commit
that was found in the repository was 2024-06-13.) If you
discover any rendering problems in this HTML version of the page,
or you believe there is a better or more up-to-date source for
the page, or you have corrections or improvements to the
information in this COLOPHON (which is not part of the original
manual page), send a mail to [email protected]
perf 2023-01-19 PERF-PROBE(1)
Pages that refer to this page: perf(1), perf-buildid-cache(1)
|
HTML rendering created 2024-06-26 by Michael Kerrisk, author of The Linux Programming Interface. For details of in-depth Linux/UNIX system programming training courses that I teach, look here. Hosting by jambit GmbH. |
|