License cleanup: add SPDX GPL-2.0 license identifier to files with no license
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.
By default all files without license information are under the default
license of the kernel, which is GPL version 2.
Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.
This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.
How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,
Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.
The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.
The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.
Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).
All documentation files were explicitly excluded.
The following heuristics were used to determine which SPDX license
identifiers to apply.
- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.
For non */uapi/* files that summary was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139
and resulted in the first patch in this series.
If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930
and resulted in the second patch in this series.
- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:
SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1
and that resulted in the third patch in this series.
- when the two scanners agreed on the detected license(s), that became
the concluded license(s).
- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.
- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).
- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.
- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.
In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.
Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.
Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.
In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.
Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct
This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.
These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-11-01 15:07:57 +01:00
|
|
|
|
2008-11-25 21:07:04 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
2008-11-25 21:07:04 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-03-25 14:22:05 -07:00
|
|
|
|
include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files. percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.
percpu.h -> slab.h dependency is about to be removed. Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability. As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.
http://userweb.kernel.org/~tj/misc/slabh-sweep.py
The script does the followings.
* Scan files for gfp and slab usages and update includes such that
only the necessary includes are there. ie. if only gfp is used,
gfp.h, if slab is used, slab.h.
* When the script inserts a new include, it looks at the include
blocks and try to put the new include such that its order conforms
to its surrounding. It's put in the include block which contains
core kernel includes, in the same order that the rest are ordered -
alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
doesn't seem to be any matching order.
* If the script can't find a place to put a new include (mostly
because the file doesn't have fitting include block), it prints out
an error message indicating which .h file needs to be added to the
file.
The conversion was done in the following steps.
1. The initial automatic conversion of all .c files updated slightly
over 4000 files, deleting around 700 includes and adding ~480 gfp.h
and ~3000 slab.h inclusions. The script emitted errors for ~400
files.
2. Each error was manually checked. Some didn't need the inclusion,
some needed manual addition while adding it to implementation .h or
embedding .c file was more appropriate for others. This step added
inclusions to around 150 files.
3. The script was run again and the output was compared to the edits
from #2 to make sure no file was left behind.
4. Several build tests were done and a couple of problems were fixed.
e.g. lib/decompress_*.c used malloc/free() wrappers around slab
APIs requiring slab.h to be added manually.
5. The script was run on all .h files but without automatically
editing them as sprinkling gfp.h and slab.h inclusions around .h
files could easily lead to inclusion dependency hell. Most gfp.h
inclusion directives were ignored as stuff from gfp.h was usually
wildly available and often used in preprocessor macros. Each
slab.h inclusion directive was examined and added manually as
necessary.
6. percpu.h was updated not to include slab.h.
7. Build test were done on the following configurations and failures
were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my
distributed build env didn't work with gcov compiles) and a few
more options had to be turned off depending on archs to make things
build (like ipr on powerpc/64 which failed due to missing writeq).
* x86 and x86_64 UP and SMP allmodconfig and a custom test config.
* powerpc and powerpc64 SMP allmodconfig
* sparc and sparc64 SMP allmodconfig
* ia64 SMP allmodconfig
* s390 SMP allmodconfig
* alpha SMP allmodconfig
* um on x86_64 SMP allmodconfig
8. percpu.h modifications were reverted so that it could be applied as
a separate patch and serve as bisection point.
Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
2010-03-24 17:04:11 +09:00
|
|
|
|
2008-11-25 21:07:04 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-12-23 23:24:12 -05:00
|
|
|
|
2008-11-25 21:07:04 +01:00
|
|
|
|
2025-11-14 14:22:30 -05:00
|
|
|
|
2010-09-14 18:58:33 -04:00
|
|
|
|
|
|
|
|
|
2025-11-14 14:22:32 -05:00
|
|
|
|
2025-11-25 10:47:51 -05:00
|
|
|
|
2025-11-14 14:22:32 -05:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
|
|
|
|
|
2010-09-07 16:53:44 +02:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
2010-02-26 17:08:16 -05:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
|
|
|
|
|
2025-08-20 19:55:22 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-11-25 17:34:25 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
2010-08-11 12:47:59 +09:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
|
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
2025-08-20 19:55:22 -04:00
|
|
|
|
2025-11-25 17:34:25 +08:00
|
|
|
|
2025-08-20 19:55:22 -04:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
|
|
|
|
|
2008-11-26 00:57:25 +01:00
|
|
|
|
2008-11-25 21:07:04 +01:00
|
|
|
|
2016-12-08 19:28:28 -05:00
|
|
|
|
2013-01-16 10:49:37 -05:00
|
|
|
|
2008-11-25 21:07:04 +01:00
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
2008-11-28 00:42:46 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-12-03 02:30:37 +01:00
|
|
|
|
|
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-09-07 16:53:44 +02:00
|
|
|
|
|
|
|
|
|
2014-05-20 17:10:51 -05:00
|
|
|
|
|
|
|
|
|
function_graph: Support recording and printing the return value of function
Analyzing system call failures with the function_graph tracer can be a
time-consuming process, particularly when locating the kernel function
that first returns an error in the trace logs. This change aims to
simplify the process by recording the function return value to the
'retval' member of 'ftrace_graph_ret' and printing it when outputting
the trace log.
We have introduced new trace options: funcgraph-retval and
funcgraph-retval-hex. The former controls whether to display the return
value, while the latter controls the display format.
Please note that even if a function's return type is void, a return
value will still be printed. You can simply ignore it.
This patch only establishes the fundamental infrastructure. Subsequent
patches will make this feature available on some commonly used processor
architectures.
Here is an example:
I attempted to attach the demo process to a cpu cgroup, but it failed:
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
-bash: echo: write error: Invalid argument
The strace logs indicate that the write system call returned -EINVAL(-22):
...
write(1, "273\n", 4) = -1 EINVAL (Invalid argument)
...
To capture trace logs during a write system call, use the following
commands:
cd /sys/kernel/debug/tracing/
echo 0 > tracing_on
echo > trace
echo *sys_write > set_graph_function
echo *spin* > set_graph_notrace
echo *rcu* >> set_graph_notrace
echo *alloc* >> set_graph_notrace
echo preempt* >> set_graph_notrace
echo kfree* >> set_graph_notrace
echo $$ > set_ftrace_pid
echo function_graph > current_tracer
echo 1 > options/funcgraph-retval
echo 0 > options/funcgraph-retval-hex
echo 1 > tracing_on
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
echo 0 > tracing_on
cat trace > ~/trace.log
To locate the root cause, search for error code -22 directly in the file
trace.log and identify the first function that returned -22. Once you
have identified this function, examine its code to determine the root
cause.
For example, in the trace log below, cpu_cgroup_can_attach
returned -22 first, so we can focus our analysis on this function to
identify the root cause.
...
1) | cgroup_migrate() {
1) 0.651 us | cgroup_migrate_add_task(); /* = 0xffff93fcfd346c00 */
1) | cgroup_migrate_execute() {
1) | cpu_cgroup_can_attach() {
1) | cgroup_taskset_first() {
1) 0.732 us | cgroup_taskset_next(); /* = 0xffff93fc8fb20000 */
1) 1.232 us | } /* cgroup_taskset_first = 0xffff93fc8fb20000 */
1) 0.380 us | sched_rt_can_attach(); /* = 0x0 */
1) 2.335 us | } /* cpu_cgroup_can_attach = -22 */
1) 4.369 us | } /* cgroup_migrate_execute = -22 */
1) 7.143 us | } /* cgroup_migrate = -22 */
...
Link: https://lkml.kernel.org/r/1fc502712c981e0e6742185ba242992170ac9da8.1680954589.git.pengdonglin@sangfor.com.cn
Tested-by: Florian Kauer <florian.kauer@linutronix.de>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Donglin Peng <pengdonglin@sangfor.com.cn>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-04-08 05:42:15 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
function_graph: Support recording and printing the return value of function
Analyzing system call failures with the function_graph tracer can be a
time-consuming process, particularly when locating the kernel function
that first returns an error in the trace logs. This change aims to
simplify the process by recording the function return value to the
'retval' member of 'ftrace_graph_ret' and printing it when outputting
the trace log.
We have introduced new trace options: funcgraph-retval and
funcgraph-retval-hex. The former controls whether to display the return
value, while the latter controls the display format.
Please note that even if a function's return type is void, a return
value will still be printed. You can simply ignore it.
This patch only establishes the fundamental infrastructure. Subsequent
patches will make this feature available on some commonly used processor
architectures.
Here is an example:
I attempted to attach the demo process to a cpu cgroup, but it failed:
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
-bash: echo: write error: Invalid argument
The strace logs indicate that the write system call returned -EINVAL(-22):
...
write(1, "273\n", 4) = -1 EINVAL (Invalid argument)
...
To capture trace logs during a write system call, use the following
commands:
cd /sys/kernel/debug/tracing/
echo 0 > tracing_on
echo > trace
echo *sys_write > set_graph_function
echo *spin* > set_graph_notrace
echo *rcu* >> set_graph_notrace
echo *alloc* >> set_graph_notrace
echo preempt* >> set_graph_notrace
echo kfree* >> set_graph_notrace
echo $$ > set_ftrace_pid
echo function_graph > current_tracer
echo 1 > options/funcgraph-retval
echo 0 > options/funcgraph-retval-hex
echo 1 > tracing_on
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
echo 0 > tracing_on
cat trace > ~/trace.log
To locate the root cause, search for error code -22 directly in the file
trace.log and identify the first function that returned -22. Once you
have identified this function, examine its code to determine the root
cause.
For example, in the trace log below, cpu_cgroup_can_attach
returned -22 first, so we can focus our analysis on this function to
identify the root cause.
...
1) | cgroup_migrate() {
1) 0.651 us | cgroup_migrate_add_task(); /* = 0xffff93fcfd346c00 */
1) | cgroup_migrate_execute() {
1) | cpu_cgroup_can_attach() {
1) | cgroup_taskset_first() {
1) 0.732 us | cgroup_taskset_next(); /* = 0xffff93fc8fb20000 */
1) 1.232 us | } /* cgroup_taskset_first = 0xffff93fc8fb20000 */
1) 0.380 us | sched_rt_can_attach(); /* = 0x0 */
1) 2.335 us | } /* cpu_cgroup_can_attach = -22 */
1) 4.369 us | } /* cgroup_migrate_execute = -22 */
1) 7.143 us | } /* cgroup_migrate = -22 */
...
Link: https://lkml.kernel.org/r/1fc502712c981e0e6742185ba242992170ac9da8.1680954589.git.pengdonglin@sangfor.com.cn
Tested-by: Florian Kauer <florian.kauer@linutronix.de>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Donglin Peng <pengdonglin@sangfor.com.cn>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-04-08 05:42:15 -07:00
|
|
|
|
2015-09-29 19:06:50 -04:00
|
|
|
|
|
|
|
|
|
2018-11-23 13:06:07 -05:00
|
|
|
|
2008-11-25 21:07:04 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-05-20 17:10:51 -05:00
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
2015-09-29 19:06:50 -04:00
|
|
|
|
2025-11-14 14:22:31 -05:00
|
|
|
|
2008-11-25 21:07:04 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-11-11 18:24:09 -05:00
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
2025-11-11 18:24:09 -05:00
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
2011-06-03 16:58:48 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-11-06 14:50:06 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-06-03 16:58:48 +02:00
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2015-09-30 09:42:05 -04:00
|
|
|
|
|
|
|
|
|
2008-11-25 21:07:04 +01:00
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
|
|
|
|
|
2009-07-29 18:59:58 +02:00
|
|
|
|
|
|
|
|
|
2019-12-13 13:58:57 -05:00
|
|
|
|
2009-07-29 18:59:58 +02:00
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
2009-07-29 18:59:58 +02:00
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-07-29 18:59:58 +02:00
|
|
|
|
|
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-09-11 09:00:26 +08:00
|
|
|
|
2009-07-29 18:59:58 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-11-25 17:34:25 +08:00
|
|
|
|
|
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-11-25 17:34:25 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
2025-11-25 17:34:25 +08:00
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-11-25 17:34:25 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-10-10 19:40:20 -04:00
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-11-25 17:34:25 +08:00
|
|
|
|
|
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-11-14 14:22:30 -05:00
|
|
|
|
2010-09-14 18:58:33 -04:00
|
|
|
|
2011-06-14 19:02:29 -04:00
|
|
|
|
2010-09-14 18:58:33 -04:00
|
|
|
|
|
|
|
|
|
2025-11-14 14:22:30 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-09-30 08:03:42 +08:00
|
|
|
|
2010-09-14 18:58:33 -04:00
|
|
|
|
|
|
|
|
|
2024-09-14 17:48:08 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-07-29 18:59:58 +02:00
|
|
|
|
2024-06-03 15:07:22 -04:00
|
|
|
|
2024-06-03 15:07:12 -04:00
|
|
|
|
2024-09-14 17:48:08 -04:00
|
|
|
|
2021-01-25 20:45:08 +01:00
|
|
|
|
fgraph: Remove unnecessary disabling of interrupts and recursion
The function graph tracer disables interrupts as well as prevents
recursion via NMIs when recording the graph tracer code. There's no reason
to do this today. That disabling goes back to 2008 when the function graph
tracer was first introduced and recursion protection wasn't part of the
code.
Today, there's no reason to disable interrupts or prevent the code from
recursing as the infrastructure can easily handle it.
Before this change:
~# echo function_graph > /sys/kernel/tracing/current_tracer
~# perf stat -r 10 ./hackbench 10
Time: 4.240
Time: 4.236
Time: 4.106
Time: 4.014
Time: 4.314
Time: 3.830
Time: 4.063
Time: 4.323
Time: 3.763
Time: 3.727
Performance counter stats for '/work/c/hackbench 10' (10 runs):
33,937.20 msec task-clock # 7.008 CPUs utilized ( +- 1.85% )
18,220 context-switches # 536.874 /sec ( +- 6.41% )
624 cpu-migrations # 18.387 /sec ( +- 9.07% )
11,319 page-faults # 333.528 /sec ( +- 1.97% )
76,657,643,617 cycles # 2.259 GHz ( +- 0.40% )
141,403,302,768 instructions # 1.84 insn per cycle ( +- 0.37% )
25,518,463,888 branches # 751.932 M/sec ( +- 0.35% )
156,151,050 branch-misses # 0.61% of all branches ( +- 0.63% )
4.8423 +- 0.0892 seconds time elapsed ( +- 1.84% )
After this change:
~# echo function_graph > /sys/kernel/tracing/current_tracer
~# perf stat -r 10 ./hackbench 10
Time: 3.340
Time: 3.192
Time: 3.129
Time: 2.579
Time: 2.589
Time: 2.798
Time: 2.791
Time: 2.955
Time: 3.044
Time: 3.065
Performance counter stats for './hackbench 10' (10 runs):
24,416.30 msec task-clock # 6.996 CPUs utilized ( +- 2.74% )
16,764 context-switches # 686.590 /sec ( +- 5.85% )
469 cpu-migrations # 19.208 /sec ( +- 6.14% )
11,519 page-faults # 471.775 /sec ( +- 1.92% )
53,895,628,450 cycles # 2.207 GHz ( +- 0.52% )
105,552,664,638 instructions # 1.96 insn per cycle ( +- 0.47% )
17,808,672,667 branches # 729.376 M/sec ( +- 0.48% )
133,075,435 branch-misses # 0.75% of all branches ( +- 0.59% )
3.490 +- 0.112 seconds time elapsed ( +- 3.22% )
Also removed unneeded "unlikely()" around the retaddr code.
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Link: https://lore.kernel.org/20241223184941.204074053@goodmis.org
Fixes: 9cd2992f2d6c8 ("fgraph: Have set_graph_notrace only affect function_graph tracer") # Performance only
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2024-12-23 13:46:19 -05:00
|
|
|
|
2009-07-29 18:59:58 +02:00
|
|
|
|
2024-06-03 15:07:22 -04:00
|
|
|
|
2018-11-14 13:14:58 -05:00
|
|
|
|
|
|
|
|
|
2019-07-30 22:08:50 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2018-11-14 13:14:58 -05:00
|
|
|
|
2025-02-08 00:15:11 -05:00
|
|
|
|
2018-11-14 13:14:58 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-06-03 15:07:20 -04:00
|
|
|
|
2016-12-08 19:28:28 -05:00
|
|
|
|
|
|
|
|
|
2025-11-14 14:22:30 -05:00
|
|
|
|
2009-07-29 18:59:58 +02:00
|
|
|
|
|
|
|
|
|
2025-11-14 14:22:32 -05:00
|
|
|
|
|
|
|
|
|
2024-09-14 17:48:08 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-11-14 14:22:32 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-09-14 17:48:07 -04:00
|
|
|
|
2024-09-14 17:48:08 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-09-14 17:48:07 -04:00
|
|
|
|
2016-06-17 22:44:54 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-05-05 17:21:11 -04:00
|
|
|
|
|
|
|
|
|
2025-11-11 18:24:09 -05:00
|
|
|
|
2025-05-05 17:21:11 -04:00
|
|
|
|
2025-11-25 17:34:25 +08:00
|
|
|
|
|
|
|
|
|
2025-05-05 17:21:11 -04:00
|
|
|
|
|
|
|
|
|
2009-07-29 18:59:58 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-09-23 14:00:52 +02:00
|
|
|
|
|
|
|
|
|
2021-01-25 20:45:08 +01:00
|
|
|
|
2010-09-23 14:00:52 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-01-25 20:45:08 +01:00
|
|
|
|
2025-01-21 19:44:36 -05:00
|
|
|
|
2010-09-23 14:00:52 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-01-25 20:45:08 +01:00
|
|
|
|
2010-09-23 14:00:52 +02:00
|
|
|
|
2021-01-25 20:45:08 +01:00
|
|
|
|
2010-09-23 14:00:52 +02:00
|
|
|
|
|
|
|
|
|
2010-04-02 19:01:22 +02:00
|
|
|
|
2025-01-21 19:44:36 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-07-29 18:59:58 +02:00
|
|
|
|
|
|
|
|
|
2019-12-13 13:58:57 -05:00
|
|
|
|
2009-07-29 18:59:58 +02:00
|
|
|
|
|
|
|
|
|
2009-09-02 14:17:06 -04:00
|
|
|
|
2021-01-25 20:45:08 +01:00
|
|
|
|
2009-07-29 18:59:58 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-01-21 19:44:36 -05:00
|
|
|
|
|
|
|
|
|
2024-09-11 09:00:26 +08:00
|
|
|
|
2009-07-29 18:59:58 +02:00
|
|
|
|
|
|
|
|
|
2025-11-14 14:22:32 -05:00
|
|
|
|
|
|
|
|
|
2024-09-14 17:48:08 -04:00
|
|
|
|
|
|
|
|
|
2024-09-14 17:48:07 -04:00
|
|
|
|
2025-11-14 14:22:32 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-09-14 17:48:07 -04:00
|
|
|
|
|
|
|
|
|
2024-09-14 17:48:08 -04:00
|
|
|
|
2024-09-14 17:48:07 -04:00
|
|
|
|
|
|
|
|
|
2024-06-03 15:07:11 -04:00
|
|
|
|
2024-12-26 14:12:09 +09:00
|
|
|
|
2009-07-29 18:59:58 +02:00
|
|
|
|
2024-06-03 15:07:22 -04:00
|
|
|
|
2024-06-03 15:07:12 -04:00
|
|
|
|
2024-09-14 17:48:08 -04:00
|
|
|
|
2021-01-25 20:45:08 +01:00
|
|
|
|
2025-01-21 19:44:36 -05:00
|
|
|
|
2024-09-14 17:48:08 -04:00
|
|
|
|
2009-07-29 18:59:58 +02:00
|
|
|
|
2025-01-21 19:44:36 -05:00
|
|
|
|
|
|
|
|
|
2024-06-03 15:07:20 -04:00
|
|
|
|
tracing/fgraph: Fix set_graph_function from showing interrupts
The tracefs file set_graph_function is used to only function graph functions
that are listed in that file (or all functions if the file is empty). The
way this is implemented is that the function graph tracer looks at every
function, and if the current depth is zero and the function matches
something in the file then it will trace that function. When other functions
are called, the depth will be greater than zero (because the original
function will be at depth zero), and all functions will be traced where the
depth is greater than zero.
The issue is that when a function is first entered, and the handler that
checks this logic is called, the depth is set to zero. If an interrupt comes
in and a function in the interrupt handler is traced, its depth will be
greater than zero and it will automatically be traced, even if the original
function was not. But because the logic only looks at depth it may trace
interrupts when it should not be.
The recent design change of the function graph tracer to fix other bugs
caused the depth to be zero while the function graph callback handler is
being called for a longer time, widening the race of this happening. This
bug was actually there for a longer time, but because the race window was so
small it seldom happened. The Fixes tag below is for the commit that widen
the race window, because that commit belongs to a series that will also help
fix the original bug.
Cc: stable@kernel.org
Fixes: 39eb456dacb5 ("function_graph: Use new curr_ret_depth to manage depth instead of curr_ret_stack")
Reported-by: Joe Lawrence <joe.lawrence@redhat.com>
Tested-by: Joe Lawrence <joe.lawrence@redhat.com>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2018-11-29 08:50:27 -05:00
|
|
|
|
2024-06-03 15:07:22 -04:00
|
|
|
|
|
|
|
|
|
2018-11-14 13:14:58 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-09-14 17:48:08 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-11-14 14:22:32 -05:00
|
|
|
|
2024-09-14 17:48:08 -04:00
|
|
|
|
2025-01-21 19:44:36 -05:00
|
|
|
|
2024-09-14 17:48:07 -04:00
|
|
|
|
2025-05-05 17:21:11 -04:00
|
|
|
|
|
|
|
|
|
2009-07-29 18:59:58 +02:00
|
|
|
|
|
|
|
|
|
2024-06-03 15:07:11 -04:00
|
|
|
|
2024-12-26 14:12:09 +09:00
|
|
|
|
|
|
|
|
|
2010-02-25 15:36:43 -08:00
|
|
|
|
2024-09-14 17:48:08 -04:00
|
|
|
|
2025-11-14 14:22:32 -05:00
|
|
|
|
2024-09-14 17:48:08 -04:00
|
|
|
|
|
|
|
|
|
2024-06-03 15:07:20 -04:00
|
|
|
|
tracing/fgraph: Fix set_graph_function from showing interrupts
The tracefs file set_graph_function is used to only function graph functions
that are listed in that file (or all functions if the file is empty). The
way this is implemented is that the function graph tracer looks at every
function, and if the current depth is zero and the function matches
something in the file then it will trace that function. When other functions
are called, the depth will be greater than zero (because the original
function will be at depth zero), and all functions will be traced where the
depth is greater than zero.
The issue is that when a function is first entered, and the handler that
checks this logic is called, the depth is set to zero. If an interrupt comes
in and a function in the interrupt handler is traced, its depth will be
greater than zero and it will automatically be traced, even if the original
function was not. But because the logic only looks at depth it may trace
interrupts when it should not be.
The recent design change of the function graph tracer to fix other bugs
caused the depth to be zero while the function graph callback handler is
being called for a longer time, widening the race of this happening. This
bug was actually there for a longer time, but because the race window was so
small it seldom happened. The Fixes tag below is for the commit that widen
the race window, because that commit belongs to a series that will also help
fix the original bug.
Cc: stable@kernel.org
Fixes: 39eb456dacb5 ("function_graph: Use new curr_ret_depth to manage depth instead of curr_ret_stack")
Reported-by: Joe Lawrence <joe.lawrence@redhat.com>
Tested-by: Joe Lawrence <joe.lawrence@redhat.com>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2018-11-29 08:50:27 -05:00
|
|
|
|
2018-11-14 13:14:58 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-09-14 17:48:08 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-11-14 14:22:32 -05:00
|
|
|
|
|
|
|
|
|
2024-09-14 17:48:08 -04:00
|
|
|
|
2010-02-25 15:36:43 -08:00
|
|
|
|
2025-01-21 19:44:36 -05:00
|
|
|
|
2010-02-25 15:36:43 -08:00
|
|
|
|
|
|
|
|
|
2024-12-26 14:12:09 +09:00
|
|
|
|
2010-02-25 15:36:43 -08:00
|
|
|
|
|
|
|
|
|
2018-11-15 14:06:47 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-06-03 15:07:16 -04:00
|
|
|
|
2024-06-03 15:07:12 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-06-03 15:07:16 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-06-03 15:07:12 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-06-03 15:07:16 -04:00
|
|
|
|
2024-06-03 15:07:12 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-06-03 15:07:16 -04:00
|
|
|
|
2024-06-03 15:07:12 -04:00
|
|
|
|
|
|
|
|
|
2008-11-25 21:07:04 +01:00
|
|
|
|
|
|
|
|
|
2009-07-29 18:59:58 +02:00
|
|
|
|
|
|
|
|
|
2025-11-11 18:24:09 -05:00
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-06-03 15:07:12 -04:00
|
|
|
|
2010-02-25 15:36:43 -08:00
|
|
|
|
2024-06-03 15:07:12 -04:00
|
|
|
|
2010-02-25 15:36:43 -08:00
|
|
|
|
2024-06-03 15:07:12 -04:00
|
|
|
|
|
|
|
|
|
2025-11-14 14:22:30 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-11-14 14:22:32 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-02-27 13:58:07 -05:00
|
|
|
|
2024-06-03 15:07:12 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-11-26 00:16:26 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-11-25 21:07:04 +01:00
|
|
|
|
|
|
|
|
|
fgraph: Do not enable function_graph tracer when setting funcgraph-args
When setting the funcgraph-args option when function graph tracer is net
enabled, it incorrectly enables it. Worse, it unregisters itself when it
was never registered. Then when it gets enabled again, it will register
itself a second time causing a WARNing.
~# echo 1 > /sys/kernel/tracing/options/funcgraph-args
~# head -20 /sys/kernel/tracing/trace
# tracer: nop
#
# entries-in-buffer/entries-written: 813/26317372 #P:8
#
# _-----=> irqs-off/BH-disabled
# / _----=> need-resched
# | / _---=> hardirq/softirq
# || / _--=> preempt-depth
# ||| / _-=> migrate-disable
# |||| / delay
# TASK-PID CPU# ||||| TIMESTAMP FUNCTION
# | | | ||||| | |
<idle>-0 [007] d..4. 358.966010: 7) 1.692 us | fetch_next_timer_interrupt(basej=4294981640, basem=357956000000, base_local=0xffff88823c3ae040, base_global=0xffff88823c3af300, tevt=0xffff888100e47cb8);
<idle>-0 [007] d..4. 358.966012: 7) | tmigr_cpu_deactivate(nextexp=357988000000) {
<idle>-0 [007] d..4. 358.966013: 7) | _raw_spin_lock(lock=0xffff88823c3b2320) {
<idle>-0 [007] d..4. 358.966014: 7) 0.981 us | preempt_count_add(val=1);
<idle>-0 [007] d..5. 358.966017: 7) 1.058 us | do_raw_spin_lock(lock=0xffff88823c3b2320);
<idle>-0 [007] d..4. 358.966019: 7) 5.824 us | }
<idle>-0 [007] d..5. 358.966021: 7) | tmigr_inactive_up(group=0xffff888100cb9000, child=0x0, data=0xffff888100e47bc0) {
<idle>-0 [007] d..5. 358.966022: 7) | tmigr_update_events(group=0xffff888100cb9000, child=0x0, data=0xffff888100e47bc0) {
Notice the "tracer: nop" at the top there. The current tracer is the "nop"
tracer, but the content is obviously the function graph tracer.
Enabling function graph tracing will cause it to register again and
trigger a warning in the accounting:
~# echo function_graph > /sys/kernel/tracing/current_tracer
-bash: echo: write error: Device or resource busy
With the dmesg of:
------------[ cut here ]------------
WARNING: CPU: 7 PID: 1095 at kernel/trace/ftrace.c:3509 ftrace_startup_subops+0xc1e/0x1000
Modules linked in: kvm_intel kvm irqbypass
CPU: 7 UID: 0 PID: 1095 Comm: bash Not tainted 6.16.0-rc2-test-00006-gea03de4105d3 #24 PREEMPT
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
RIP: 0010:ftrace_startup_subops+0xc1e/0x1000
Code: 48 b8 22 01 00 00 00 00 ad de 49 89 84 24 88 01 00 00 8b 44 24 08 89 04 24 e9 c3 f7 ff ff c7 04 24 ed ff ff ff e9 b7 f7 ff ff <0f> 0b c7 04 24 f0 ff ff ff e9 a9 f7 ff ff c7 04 24 f4 ff ff ff e9
RSP: 0018:ffff888133cff948 EFLAGS: 00010202
RAX: 0000000000000001 RBX: 1ffff1102679ff31 RCX: 0000000000000000
RDX: 1ffffffff0b27a60 RSI: ffffffff8593d2f0 RDI: ffffffff85941140
RBP: 00000000000c2041 R08: ffffffffffffffff R09: ffffed1020240221
R10: ffff88810120110f R11: ffffed1020240214 R12: ffffffff8593d2f0
R13: ffffffff8593d300 R14: ffffffff85941140 R15: ffffffff85631100
FS: 00007f7ec6f28740(0000) GS:ffff8882b5251000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7ec6f181c0 CR3: 000000012f1d0005 CR4: 0000000000172ef0
Call Trace:
<TASK>
? __pfx_ftrace_startup_subops+0x10/0x10
? find_held_lock+0x2b/0x80
? ftrace_stub_direct_tramp+0x10/0x10
? ftrace_stub_direct_tramp+0x10/0x10
? trace_preempt_on+0xd0/0x110
? __pfx_trace_graph_entry_args+0x10/0x10
register_ftrace_graph+0x4d2/0x1020
? tracing_reset_online_cpus+0x14b/0x1e0
? __pfx_register_ftrace_graph+0x10/0x10
? ring_buffer_record_enable+0x16/0x20
? tracing_reset_online_cpus+0x153/0x1e0
? __pfx_tracing_reset_online_cpus+0x10/0x10
? __pfx_trace_graph_return+0x10/0x10
graph_trace_init+0xfd/0x160
tracing_set_tracer+0x500/0xa80
? __pfx_tracing_set_tracer+0x10/0x10
? lock_release+0x181/0x2d0
? _copy_from_user+0x26/0xa0
tracing_set_trace_write+0x132/0x1e0
? __pfx_tracing_set_trace_write+0x10/0x10
? ftrace_graph_func+0xcc/0x140
? ftrace_stub_direct_tramp+0x10/0x10
? ftrace_stub_direct_tramp+0x10/0x10
? ftrace_stub_direct_tramp+0x10/0x10
vfs_write+0x1d0/0xe90
? __pfx_vfs_write+0x10/0x10
Have the setting of the funcgraph-args check if function_graph tracer is
the current tracer of the instance, and if not, do nothing, as there's
nothing to do (the option is checked when function_graph tracing starts).
Cc: stable@vger.kernel.org
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Link: https://lore.kernel.org/20250618073801.057ea636@gandalf.local.home
Fixes: c7a60a733c373 ("ftrace: Have funcgraph-args take affect during tracing")
Closes: https://lore.kernel.org/all/4ab1a7bdd0174ab09c7b0d68cdbff9a4@huawei.com/
Reported-by: Changbin Du <changbin.du@huawei.com>
Tested-by: Changbin Du <changbin.du@huawei.com>
Reviewed-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2025-06-18 07:38:01 -04:00
|
|
|
|
|
|
|
|
|
2025-02-27 13:58:07 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-11-25 21:07:04 +01:00
|
|
|
|
|
|
|
|
|
2025-11-14 14:22:30 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-11-14 14:22:32 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-11-26 00:16:26 -05:00
|
|
|
|
2024-06-03 15:07:12 -04:00
|
|
|
|
2008-11-25 21:07:04 +01:00
|
|
|
|
|
|
|
|
|
2014-07-18 18:07:49 -04:00
|
|
|
|
2014-07-18 15:17:27 +04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-07-28 20:26:06 +08:00
|
|
|
|
2008-11-28 00:42:46 +01:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2008-11-28 00:42:46 +01:00
|
|
|
|
2008-11-28 09:55:16 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2008-11-28 00:42:46 +01:00
|
|
|
|
|
|
|
|
|
2008-12-03 02:30:37 +01:00
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2008-12-03 02:30:37 +01:00
|
|
|
|
2009-03-16 19:20:15 -04:00
|
|
|
|
2008-12-03 02:30:37 +01:00
|
|
|
|
2025-06-17 15:21:09 +00:00
|
|
|
|
2009-03-16 19:20:15 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-12-03 02:30:37 +01:00
|
|
|
|
2009-03-16 19:20:15 -04:00
|
|
|
|
2008-12-03 02:30:37 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
|
|
|
|
|
2008-12-03 02:30:37 +01:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2008-12-03 02:30:37 +01:00
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
|
|
|
|
|
2008-12-03 02:30:37 +01:00
|
|
|
|
|
|
|
|
|
2008-11-28 00:42:46 +01:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2009-09-11 00:30:26 -04:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
|
|
|
|
|
2019-01-01 23:46:12 +08:00
|
|
|
|
2009-09-11 00:30:26 -04:00
|
|
|
|
|
|
|
|
|
2008-11-26 00:57:25 +01:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
2008-11-26 00:57:25 +01:00
|
|
|
|
2008-11-28 09:55:16 +01:00
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
2008-11-26 00:16:26 -05:00
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2008-11-26 00:57:25 +01:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
2008-11-26 00:16:26 -05:00
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2008-11-28 09:55:16 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-11-26 00:57:25 +01:00
|
|
|
|
|
|
|
|
|
2009-02-06 18:30:44 +01:00
|
|
|
|
|
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
2008-11-26 00:57:25 +01:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
|
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
|
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
2012-06-27 20:46:14 -04:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-01-09 18:53:48 -05:00
|
|
|
|
ring-buffer: Add place holder recording of dropped events
Currently, when the ring buffer drops events, it does not record
the fact that it did so. It does inform the writer that the event
was dropped by returning a NULL event, but it does not put in any
place holder where the event was dropped.
This is not a trivial thing to add because the ring buffer mostly
runs in overwrite (flight recorder) mode. That is, when the ring
buffer is full, new data will overwrite old data.
In a produce/consumer mode, where new data is simply dropped when
the ring buffer is full, it is trivial to add the placeholder
for dropped events. When there's more room to write new data, then
a special event can be added to notify the reader about the dropped
events.
But in overwrite mode, any new write can overwrite events. A place
holder can not be inserted into the ring buffer since there never
may be room. A reader could also come in at anytime and miss the
placeholder.
Luckily, the way the ring buffer works, the read side can find out
if events were lost or not, and how many events. Everytime a write
takes place, if it overwrites the header page (the next read) it
updates a "overrun" variable that keeps track of the number of
lost events. When a reader swaps out a page from the ring buffer,
it can record this number, perfom the swap, and then check to
see if the number changed, and take the diff if it has, which would be
the number of events dropped. This can be stored by the reader
and returned to callers of the reader.
Since the reader page swap will fail if the writer moved the head
page since the time the reader page set up the swap, this gives room
to record the overruns without worrying about races. If the reader
sets up the pages, records the overrun, than performs the swap,
if the swap succeeds, then the overrun variable has not been
updated since the setup before the swap.
For binary readers of the ring buffer, a flag is set in the header
of each sub page (sub buffer) of the ring buffer. This flag is embedded
in the size field of the data on the sub buffer, in the 31st bit (the size
can be 32 or 64 bits depending on the architecture), but only 27
bits needs to be used for the actual size (less actually).
We could add a new field in the sub buffer header to also record the
number of events dropped since the last read, but this will change the
format of the binary ring buffer a bit too much. Perhaps this change can
be made if the information on the number of events dropped is considered
important enough.
Note, the notification of dropped events is only used by consuming reads
or peeking at the ring buffer. Iterating over the ring buffer does not
keep this information because the necessary data is only available when
a page swap is made, and the iterator does not swap out pages.
Cc: Robert Richter <robert.richter@amd.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Li Zefan <lizf@cn.fujitsu.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: "Luis Claudio R. Goncalves" <lclaudio@uudg.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2010-03-31 13:21:56 -04:00
|
|
|
|
2020-01-09 18:53:48 -05:00
|
|
|
|
ring-buffer: Add place holder recording of dropped events
Currently, when the ring buffer drops events, it does not record
the fact that it did so. It does inform the writer that the event
was dropped by returning a NULL event, but it does not put in any
place holder where the event was dropped.
This is not a trivial thing to add because the ring buffer mostly
runs in overwrite (flight recorder) mode. That is, when the ring
buffer is full, new data will overwrite old data.
In a produce/consumer mode, where new data is simply dropped when
the ring buffer is full, it is trivial to add the placeholder
for dropped events. When there's more room to write new data, then
a special event can be added to notify the reader about the dropped
events.
But in overwrite mode, any new write can overwrite events. A place
holder can not be inserted into the ring buffer since there never
may be room. A reader could also come in at anytime and miss the
placeholder.
Luckily, the way the ring buffer works, the read side can find out
if events were lost or not, and how many events. Everytime a write
takes place, if it overwrites the header page (the next read) it
updates a "overrun" variable that keeps track of the number of
lost events. When a reader swaps out a page from the ring buffer,
it can record this number, perfom the swap, and then check to
see if the number changed, and take the diff if it has, which would be
the number of events dropped. This can be stored by the reader
and returned to callers of the reader.
Since the reader page swap will fail if the writer moved the head
page since the time the reader page set up the swap, this gives room
to record the overruns without worrying about races. If the reader
sets up the pages, records the overrun, than performs the swap,
if the swap succeeds, then the overrun variable has not been
updated since the setup before the swap.
For binary readers of the ring buffer, a flag is set in the header
of each sub page (sub buffer) of the ring buffer. This flag is embedded
in the size field of the data on the sub buffer, in the 31st bit (the size
can be 32 or 64 bits depending on the architecture), but only 27
bits needs to be used for the actual size (less actually).
We could add a new field in the sub buffer header to also record the
number of events dropped since the last read, but this will change the
format of the binary ring buffer a bit too much. Perhaps this change can
be made if the information on the number of events dropped is considered
important enough.
Note, the notification of dropped events is only used by consuming reads
or peeking at the ring buffer. Iterating over the ring buffer does not
keep this information because the necessary data is only available when
a page swap is made, and the iterator does not swap out pages.
Cc: Robert Richter <robert.richter@amd.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Li Zefan <lizf@cn.fujitsu.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: "Luis Claudio R. Goncalves" <lclaudio@uudg.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2010-03-31 13:21:56 -04:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-11-25 17:34:25 +08:00
|
|
|
|
2025-08-20 19:55:22 -04:00
|
|
|
|
2025-11-25 17:34:25 +08:00
|
|
|
|
2010-07-27 16:06:34 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
2009-02-06 18:30:44 +01:00
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
|
|
|
|
|
2009-02-06 18:30:44 +01:00
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-02-06 18:30:44 +01:00
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
2009-02-06 18:30:44 +01:00
|
|
|
|
|
|
|
|
|
2020-03-17 17:32:25 -04:00
|
|
|
|
2009-02-06 18:30:44 +01:00
|
|
|
|
|
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2009-02-18 04:25:25 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
|
|
|
|
|
2009-02-18 04:25:25 +01:00
|
|
|
|
|
|
|
|
|
2019-01-01 23:46:10 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-01-09 18:53:48 -05:00
|
|
|
|
2019-01-01 23:46:10 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2009-02-18 04:25:25 +01:00
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
2008-12-09 23:55:25 +01:00
|
|
|
|
2015-09-30 09:42:05 -04:00
|
|
|
|
2009-02-18 04:25:25 +01:00
|
|
|
|
2014-11-06 15:25:09 -02:00
|
|
|
|
2008-12-09 23:55:25 +01:00
|
|
|
|
2024-08-13 13:11:06 -04:00
|
|
|
|
|
|
|
|
|
2008-12-09 23:55:25 +01:00
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2008-12-09 23:55:25 +01:00
|
|
|
|
2025-10-31 11:46:11 +09:00
|
|
|
|
2011-06-03 16:58:51 +02:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
|
|
|
|
|
2009-02-18 04:25:25 +01:00
|
|
|
|
2019-01-01 23:46:10 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-06-03 16:58:51 +02:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
|
|
|
|
|
2009-09-11 00:30:26 -04:00
|
|
|
|
2011-06-03 16:58:51 +02:00
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
|
|
|
|
|
2011-06-03 16:58:51 +02:00
|
|
|
|
2014-11-06 15:25:09 -02:00
|
|
|
|
|
|
|
|
|
2025-10-31 11:46:11 +09:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-09-30 09:42:05 -04:00
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
2015-09-30 09:42:05 -04:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2008-12-09 23:55:25 +01:00
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2009-03-23 23:12:58 -04:00
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
|
|
|
|
|
2008-12-03 02:32:12 +01:00
|
|
|
|
2014-11-05 16:18:44 +09:00
|
|
|
|
2008-12-03 02:32:12 +01:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2008-12-03 02:32:12 +01:00
|
|
|
|
|
|
|
|
|
2014-11-05 16:18:44 +09:00
|
|
|
|
2008-12-03 02:32:12 +01:00
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2008-12-03 02:32:12 +01:00
|
|
|
|
2014-11-05 16:18:44 +09:00
|
|
|
|
2008-12-03 02:32:12 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-09-29 10:08:23 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2015-07-16 21:58:52 -04:00
|
|
|
|
2008-12-03 02:32:12 +01:00
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2008-12-03 02:32:12 +01:00
|
|
|
|
|
|
|
|
|
2015-07-16 21:58:52 -04:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2009-03-23 23:12:58 -04:00
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2015-09-30 09:42:05 -04:00
|
|
|
|
|
|
|
|
|
2009-03-23 23:12:58 -04:00
|
|
|
|
2011-06-03 16:58:51 +02:00
|
|
|
|
2025-10-31 11:46:11 +09:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2011-06-03 16:58:48 +02:00
|
|
|
|
|
|
|
|
|
2013-11-06 14:50:06 -05:00
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
|
|
|
|
|
2013-11-06 14:50:06 -05:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
|
|
|
|
|
2013-11-06 14:50:06 -05:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
|
|
|
|
|
2011-06-03 16:58:48 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-11-24 09:34:19 +09:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2011-06-03 16:58:48 +02:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
|
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
|
|
|
|
|
function_graph: Support recording and printing the return value of function
Analyzing system call failures with the function_graph tracer can be a
time-consuming process, particularly when locating the kernel function
that first returns an error in the trace logs. This change aims to
simplify the process by recording the function return value to the
'retval' member of 'ftrace_graph_ret' and printing it when outputting
the trace log.
We have introduced new trace options: funcgraph-retval and
funcgraph-retval-hex. The former controls whether to display the return
value, while the latter controls the display format.
Please note that even if a function's return type is void, a return
value will still be printed. You can simply ignore it.
This patch only establishes the fundamental infrastructure. Subsequent
patches will make this feature available on some commonly used processor
architectures.
Here is an example:
I attempted to attach the demo process to a cpu cgroup, but it failed:
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
-bash: echo: write error: Invalid argument
The strace logs indicate that the write system call returned -EINVAL(-22):
...
write(1, "273\n", 4) = -1 EINVAL (Invalid argument)
...
To capture trace logs during a write system call, use the following
commands:
cd /sys/kernel/debug/tracing/
echo 0 > tracing_on
echo > trace
echo *sys_write > set_graph_function
echo *spin* > set_graph_notrace
echo *rcu* >> set_graph_notrace
echo *alloc* >> set_graph_notrace
echo preempt* >> set_graph_notrace
echo kfree* >> set_graph_notrace
echo $$ > set_ftrace_pid
echo function_graph > current_tracer
echo 1 > options/funcgraph-retval
echo 0 > options/funcgraph-retval-hex
echo 1 > tracing_on
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
echo 0 > tracing_on
cat trace > ~/trace.log
To locate the root cause, search for error code -22 directly in the file
trace.log and identify the first function that returned -22. Once you
have identified this function, examine its code to determine the root
cause.
For example, in the trace log below, cpu_cgroup_can_attach
returned -22 first, so we can focus our analysis on this function to
identify the root cause.
...
1) | cgroup_migrate() {
1) 0.651 us | cgroup_migrate_add_task(); /* = 0xffff93fcfd346c00 */
1) | cgroup_migrate_execute() {
1) | cpu_cgroup_can_attach() {
1) | cgroup_taskset_first() {
1) 0.732 us | cgroup_taskset_next(); /* = 0xffff93fc8fb20000 */
1) 1.232 us | } /* cgroup_taskset_first = 0xffff93fc8fb20000 */
1) 0.380 us | sched_rt_can_attach(); /* = 0x0 */
1) 2.335 us | } /* cpu_cgroup_can_attach = -22 */
1) 4.369 us | } /* cgroup_migrate_execute = -22 */
1) 7.143 us | } /* cgroup_migrate = -22 */
...
Link: https://lkml.kernel.org/r/1fc502712c981e0e6742185ba242992170ac9da8.1680954589.git.pengdonglin@sangfor.com.cn
Tested-by: Florian Kauer <florian.kauer@linutronix.de>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Donglin Peng <pengdonglin@sangfor.com.cn>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-04-08 05:42:15 -07:00
|
|
|
|
|
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
function_graph: Support recording and printing the return value of function
Analyzing system call failures with the function_graph tracer can be a
time-consuming process, particularly when locating the kernel function
that first returns an error in the trace logs. This change aims to
simplify the process by recording the function return value to the
'retval' member of 'ftrace_graph_ret' and printing it when outputting
the trace log.
We have introduced new trace options: funcgraph-retval and
funcgraph-retval-hex. The former controls whether to display the return
value, while the latter controls the display format.
Please note that even if a function's return type is void, a return
value will still be printed. You can simply ignore it.
This patch only establishes the fundamental infrastructure. Subsequent
patches will make this feature available on some commonly used processor
architectures.
Here is an example:
I attempted to attach the demo process to a cpu cgroup, but it failed:
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
-bash: echo: write error: Invalid argument
The strace logs indicate that the write system call returned -EINVAL(-22):
...
write(1, "273\n", 4) = -1 EINVAL (Invalid argument)
...
To capture trace logs during a write system call, use the following
commands:
cd /sys/kernel/debug/tracing/
echo 0 > tracing_on
echo > trace
echo *sys_write > set_graph_function
echo *spin* > set_graph_notrace
echo *rcu* >> set_graph_notrace
echo *alloc* >> set_graph_notrace
echo preempt* >> set_graph_notrace
echo kfree* >> set_graph_notrace
echo $$ > set_ftrace_pid
echo function_graph > current_tracer
echo 1 > options/funcgraph-retval
echo 0 > options/funcgraph-retval-hex
echo 1 > tracing_on
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
echo 0 > tracing_on
cat trace > ~/trace.log
To locate the root cause, search for error code -22 directly in the file
trace.log and identify the first function that returned -22. Once you
have identified this function, examine its code to determine the root
cause.
For example, in the trace log below, cpu_cgroup_can_attach
returned -22 first, so we can focus our analysis on this function to
identify the root cause.
...
1) | cgroup_migrate() {
1) 0.651 us | cgroup_migrate_add_task(); /* = 0xffff93fcfd346c00 */
1) | cgroup_migrate_execute() {
1) | cpu_cgroup_can_attach() {
1) | cgroup_taskset_first() {
1) 0.732 us | cgroup_taskset_next(); /* = 0xffff93fc8fb20000 */
1) 1.232 us | } /* cgroup_taskset_first = 0xffff93fc8fb20000 */
1) 0.380 us | sched_rt_can_attach(); /* = 0x0 */
1) 2.335 us | } /* cpu_cgroup_can_attach = -22 */
1) 4.369 us | } /* cgroup_migrate_execute = -22 */
1) 7.143 us | } /* cgroup_migrate = -22 */
...
Link: https://lkml.kernel.org/r/1fc502712c981e0e6742185ba242992170ac9da8.1680954589.git.pengdonglin@sangfor.com.cn
Tested-by: Florian Kauer <florian.kauer@linutronix.de>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Donglin Peng <pengdonglin@sangfor.com.cn>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-04-08 05:42:15 -07:00
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-11-25 17:34:25 +08:00
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
function_graph: Support recording and printing the return value of function
Analyzing system call failures with the function_graph tracer can be a
time-consuming process, particularly when locating the kernel function
that first returns an error in the trace logs. This change aims to
simplify the process by recording the function return value to the
'retval' member of 'ftrace_graph_ret' and printing it when outputting
the trace log.
We have introduced new trace options: funcgraph-retval and
funcgraph-retval-hex. The former controls whether to display the return
value, while the latter controls the display format.
Please note that even if a function's return type is void, a return
value will still be printed. You can simply ignore it.
This patch only establishes the fundamental infrastructure. Subsequent
patches will make this feature available on some commonly used processor
architectures.
Here is an example:
I attempted to attach the demo process to a cpu cgroup, but it failed:
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
-bash: echo: write error: Invalid argument
The strace logs indicate that the write system call returned -EINVAL(-22):
...
write(1, "273\n", 4) = -1 EINVAL (Invalid argument)
...
To capture trace logs during a write system call, use the following
commands:
cd /sys/kernel/debug/tracing/
echo 0 > tracing_on
echo > trace
echo *sys_write > set_graph_function
echo *spin* > set_graph_notrace
echo *rcu* >> set_graph_notrace
echo *alloc* >> set_graph_notrace
echo preempt* >> set_graph_notrace
echo kfree* >> set_graph_notrace
echo $$ > set_ftrace_pid
echo function_graph > current_tracer
echo 1 > options/funcgraph-retval
echo 0 > options/funcgraph-retval-hex
echo 1 > tracing_on
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
echo 0 > tracing_on
cat trace > ~/trace.log
To locate the root cause, search for error code -22 directly in the file
trace.log and identify the first function that returned -22. Once you
have identified this function, examine its code to determine the root
cause.
For example, in the trace log below, cpu_cgroup_can_attach
returned -22 first, so we can focus our analysis on this function to
identify the root cause.
...
1) | cgroup_migrate() {
1) 0.651 us | cgroup_migrate_add_task(); /* = 0xffff93fcfd346c00 */
1) | cgroup_migrate_execute() {
1) | cpu_cgroup_can_attach() {
1) | cgroup_taskset_first() {
1) 0.732 us | cgroup_taskset_next(); /* = 0xffff93fc8fb20000 */
1) 1.232 us | } /* cgroup_taskset_first = 0xffff93fc8fb20000 */
1) 0.380 us | sched_rt_can_attach(); /* = 0x0 */
1) 2.335 us | } /* cpu_cgroup_can_attach = -22 */
1) 4.369 us | } /* cgroup_migrate_execute = -22 */
1) 7.143 us | } /* cgroup_migrate = -22 */
...
Link: https://lkml.kernel.org/r/1fc502712c981e0e6742185ba242992170ac9da8.1680954589.git.pengdonglin@sangfor.com.cn
Tested-by: Florian Kauer <florian.kauer@linutronix.de>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Donglin Peng <pengdonglin@sangfor.com.cn>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-04-08 05:42:15 -07:00
|
|
|
|
|
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
function_graph: Support recording and printing the return value of function
Analyzing system call failures with the function_graph tracer can be a
time-consuming process, particularly when locating the kernel function
that first returns an error in the trace logs. This change aims to
simplify the process by recording the function return value to the
'retval' member of 'ftrace_graph_ret' and printing it when outputting
the trace log.
We have introduced new trace options: funcgraph-retval and
funcgraph-retval-hex. The former controls whether to display the return
value, while the latter controls the display format.
Please note that even if a function's return type is void, a return
value will still be printed. You can simply ignore it.
This patch only establishes the fundamental infrastructure. Subsequent
patches will make this feature available on some commonly used processor
architectures.
Here is an example:
I attempted to attach the demo process to a cpu cgroup, but it failed:
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
-bash: echo: write error: Invalid argument
The strace logs indicate that the write system call returned -EINVAL(-22):
...
write(1, "273\n", 4) = -1 EINVAL (Invalid argument)
...
To capture trace logs during a write system call, use the following
commands:
cd /sys/kernel/debug/tracing/
echo 0 > tracing_on
echo > trace
echo *sys_write > set_graph_function
echo *spin* > set_graph_notrace
echo *rcu* >> set_graph_notrace
echo *alloc* >> set_graph_notrace
echo preempt* >> set_graph_notrace
echo kfree* >> set_graph_notrace
echo $$ > set_ftrace_pid
echo function_graph > current_tracer
echo 1 > options/funcgraph-retval
echo 0 > options/funcgraph-retval-hex
echo 1 > tracing_on
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
echo 0 > tracing_on
cat trace > ~/trace.log
To locate the root cause, search for error code -22 directly in the file
trace.log and identify the first function that returned -22. Once you
have identified this function, examine its code to determine the root
cause.
For example, in the trace log below, cpu_cgroup_can_attach
returned -22 first, so we can focus our analysis on this function to
identify the root cause.
...
1) | cgroup_migrate() {
1) 0.651 us | cgroup_migrate_add_task(); /* = 0xffff93fcfd346c00 */
1) | cgroup_migrate_execute() {
1) | cpu_cgroup_can_attach() {
1) | cgroup_taskset_first() {
1) 0.732 us | cgroup_taskset_next(); /* = 0xffff93fc8fb20000 */
1) 1.232 us | } /* cgroup_taskset_first = 0xffff93fc8fb20000 */
1) 0.380 us | sched_rt_can_attach(); /* = 0x0 */
1) 2.335 us | } /* cpu_cgroup_can_attach = -22 */
1) 4.369 us | } /* cgroup_migrate_execute = -22 */
1) 7.143 us | } /* cgroup_migrate = -22 */
...
Link: https://lkml.kernel.org/r/1fc502712c981e0e6742185ba242992170ac9da8.1680954589.git.pengdonglin@sangfor.com.cn
Tested-by: Florian Kauer <florian.kauer@linutronix.de>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Donglin Peng <pengdonglin@sangfor.com.cn>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-04-08 05:42:15 -07:00
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
function_graph: Support recording and printing the return value of function
Analyzing system call failures with the function_graph tracer can be a
time-consuming process, particularly when locating the kernel function
that first returns an error in the trace logs. This change aims to
simplify the process by recording the function return value to the
'retval' member of 'ftrace_graph_ret' and printing it when outputting
the trace log.
We have introduced new trace options: funcgraph-retval and
funcgraph-retval-hex. The former controls whether to display the return
value, while the latter controls the display format.
Please note that even if a function's return type is void, a return
value will still be printed. You can simply ignore it.
This patch only establishes the fundamental infrastructure. Subsequent
patches will make this feature available on some commonly used processor
architectures.
Here is an example:
I attempted to attach the demo process to a cpu cgroup, but it failed:
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
-bash: echo: write error: Invalid argument
The strace logs indicate that the write system call returned -EINVAL(-22):
...
write(1, "273\n", 4) = -1 EINVAL (Invalid argument)
...
To capture trace logs during a write system call, use the following
commands:
cd /sys/kernel/debug/tracing/
echo 0 > tracing_on
echo > trace
echo *sys_write > set_graph_function
echo *spin* > set_graph_notrace
echo *rcu* >> set_graph_notrace
echo *alloc* >> set_graph_notrace
echo preempt* >> set_graph_notrace
echo kfree* >> set_graph_notrace
echo $$ > set_ftrace_pid
echo function_graph > current_tracer
echo 1 > options/funcgraph-retval
echo 0 > options/funcgraph-retval-hex
echo 1 > tracing_on
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
echo 0 > tracing_on
cat trace > ~/trace.log
To locate the root cause, search for error code -22 directly in the file
trace.log and identify the first function that returned -22. Once you
have identified this function, examine its code to determine the root
cause.
For example, in the trace log below, cpu_cgroup_can_attach
returned -22 first, so we can focus our analysis on this function to
identify the root cause.
...
1) | cgroup_migrate() {
1) 0.651 us | cgroup_migrate_add_task(); /* = 0xffff93fcfd346c00 */
1) | cgroup_migrate_execute() {
1) | cpu_cgroup_can_attach() {
1) | cgroup_taskset_first() {
1) 0.732 us | cgroup_taskset_next(); /* = 0xffff93fc8fb20000 */
1) 1.232 us | } /* cgroup_taskset_first = 0xffff93fc8fb20000 */
1) 0.380 us | sched_rt_can_attach(); /* = 0x0 */
1) 2.335 us | } /* cpu_cgroup_can_attach = -22 */
1) 4.369 us | } /* cgroup_migrate_execute = -22 */
1) 7.143 us | } /* cgroup_migrate = -22 */
...
Link: https://lkml.kernel.org/r/1fc502712c981e0e6742185ba242992170ac9da8.1680954589.git.pengdonglin@sangfor.com.cn
Tested-by: Florian Kauer <florian.kauer@linutronix.de>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Donglin Peng <pengdonglin@sangfor.com.cn>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-04-08 05:42:15 -07:00
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
function_graph: Support recording and printing the return value of function
Analyzing system call failures with the function_graph tracer can be a
time-consuming process, particularly when locating the kernel function
that first returns an error in the trace logs. This change aims to
simplify the process by recording the function return value to the
'retval' member of 'ftrace_graph_ret' and printing it when outputting
the trace log.
We have introduced new trace options: funcgraph-retval and
funcgraph-retval-hex. The former controls whether to display the return
value, while the latter controls the display format.
Please note that even if a function's return type is void, a return
value will still be printed. You can simply ignore it.
This patch only establishes the fundamental infrastructure. Subsequent
patches will make this feature available on some commonly used processor
architectures.
Here is an example:
I attempted to attach the demo process to a cpu cgroup, but it failed:
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
-bash: echo: write error: Invalid argument
The strace logs indicate that the write system call returned -EINVAL(-22):
...
write(1, "273\n", 4) = -1 EINVAL (Invalid argument)
...
To capture trace logs during a write system call, use the following
commands:
cd /sys/kernel/debug/tracing/
echo 0 > tracing_on
echo > trace
echo *sys_write > set_graph_function
echo *spin* > set_graph_notrace
echo *rcu* >> set_graph_notrace
echo *alloc* >> set_graph_notrace
echo preempt* >> set_graph_notrace
echo kfree* >> set_graph_notrace
echo $$ > set_ftrace_pid
echo function_graph > current_tracer
echo 1 > options/funcgraph-retval
echo 0 > options/funcgraph-retval-hex
echo 1 > tracing_on
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
echo 0 > tracing_on
cat trace > ~/trace.log
To locate the root cause, search for error code -22 directly in the file
trace.log and identify the first function that returned -22. Once you
have identified this function, examine its code to determine the root
cause.
For example, in the trace log below, cpu_cgroup_can_attach
returned -22 first, so we can focus our analysis on this function to
identify the root cause.
...
1) | cgroup_migrate() {
1) 0.651 us | cgroup_migrate_add_task(); /* = 0xffff93fcfd346c00 */
1) | cgroup_migrate_execute() {
1) | cpu_cgroup_can_attach() {
1) | cgroup_taskset_first() {
1) 0.732 us | cgroup_taskset_next(); /* = 0xffff93fc8fb20000 */
1) 1.232 us | } /* cgroup_taskset_first = 0xffff93fc8fb20000 */
1) 0.380 us | sched_rt_can_attach(); /* = 0x0 */
1) 2.335 us | } /* cpu_cgroup_can_attach = -22 */
1) 4.369 us | } /* cgroup_migrate_execute = -22 */
1) 7.143 us | } /* cgroup_migrate = -22 */
...
Link: https://lkml.kernel.org/r/1fc502712c981e0e6742185ba242992170ac9da8.1680954589.git.pengdonglin@sangfor.com.cn
Tested-by: Florian Kauer <florian.kauer@linutronix.de>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Donglin Peng <pengdonglin@sangfor.com.cn>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-04-08 05:42:15 -07:00
|
|
|
|
|
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
function_graph: Support recording and printing the return value of function
Analyzing system call failures with the function_graph tracer can be a
time-consuming process, particularly when locating the kernel function
that first returns an error in the trace logs. This change aims to
simplify the process by recording the function return value to the
'retval' member of 'ftrace_graph_ret' and printing it when outputting
the trace log.
We have introduced new trace options: funcgraph-retval and
funcgraph-retval-hex. The former controls whether to display the return
value, while the latter controls the display format.
Please note that even if a function's return type is void, a return
value will still be printed. You can simply ignore it.
This patch only establishes the fundamental infrastructure. Subsequent
patches will make this feature available on some commonly used processor
architectures.
Here is an example:
I attempted to attach the demo process to a cpu cgroup, but it failed:
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
-bash: echo: write error: Invalid argument
The strace logs indicate that the write system call returned -EINVAL(-22):
...
write(1, "273\n", 4) = -1 EINVAL (Invalid argument)
...
To capture trace logs during a write system call, use the following
commands:
cd /sys/kernel/debug/tracing/
echo 0 > tracing_on
echo > trace
echo *sys_write > set_graph_function
echo *spin* > set_graph_notrace
echo *rcu* >> set_graph_notrace
echo *alloc* >> set_graph_notrace
echo preempt* >> set_graph_notrace
echo kfree* >> set_graph_notrace
echo $$ > set_ftrace_pid
echo function_graph > current_tracer
echo 1 > options/funcgraph-retval
echo 0 > options/funcgraph-retval-hex
echo 1 > tracing_on
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
echo 0 > tracing_on
cat trace > ~/trace.log
To locate the root cause, search for error code -22 directly in the file
trace.log and identify the first function that returned -22. Once you
have identified this function, examine its code to determine the root
cause.
For example, in the trace log below, cpu_cgroup_can_attach
returned -22 first, so we can focus our analysis on this function to
identify the root cause.
...
1) | cgroup_migrate() {
1) 0.651 us | cgroup_migrate_add_task(); /* = 0xffff93fcfd346c00 */
1) | cgroup_migrate_execute() {
1) | cpu_cgroup_can_attach() {
1) | cgroup_taskset_first() {
1) 0.732 us | cgroup_taskset_next(); /* = 0xffff93fc8fb20000 */
1) 1.232 us | } /* cgroup_taskset_first = 0xffff93fc8fb20000 */
1) 0.380 us | sched_rt_can_attach(); /* = 0x0 */
1) 2.335 us | } /* cpu_cgroup_can_attach = -22 */
1) 4.369 us | } /* cgroup_migrate_execute = -22 */
1) 7.143 us | } /* cgroup_migrate = -22 */
...
Link: https://lkml.kernel.org/r/1fc502712c981e0e6742185ba242992170ac9da8.1680954589.git.pengdonglin@sangfor.com.cn
Tested-by: Florian Kauer <florian.kauer@linutronix.de>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Donglin Peng <pengdonglin@sangfor.com.cn>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-04-08 05:42:15 -07:00
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2026-01-12 10:16:01 +08:00
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
function_graph: Support recording and printing the return value of function
Analyzing system call failures with the function_graph tracer can be a
time-consuming process, particularly when locating the kernel function
that first returns an error in the trace logs. This change aims to
simplify the process by recording the function return value to the
'retval' member of 'ftrace_graph_ret' and printing it when outputting
the trace log.
We have introduced new trace options: funcgraph-retval and
funcgraph-retval-hex. The former controls whether to display the return
value, while the latter controls the display format.
Please note that even if a function's return type is void, a return
value will still be printed. You can simply ignore it.
This patch only establishes the fundamental infrastructure. Subsequent
patches will make this feature available on some commonly used processor
architectures.
Here is an example:
I attempted to attach the demo process to a cpu cgroup, but it failed:
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
-bash: echo: write error: Invalid argument
The strace logs indicate that the write system call returned -EINVAL(-22):
...
write(1, "273\n", 4) = -1 EINVAL (Invalid argument)
...
To capture trace logs during a write system call, use the following
commands:
cd /sys/kernel/debug/tracing/
echo 0 > tracing_on
echo > trace
echo *sys_write > set_graph_function
echo *spin* > set_graph_notrace
echo *rcu* >> set_graph_notrace
echo *alloc* >> set_graph_notrace
echo preempt* >> set_graph_notrace
echo kfree* >> set_graph_notrace
echo $$ > set_ftrace_pid
echo function_graph > current_tracer
echo 1 > options/funcgraph-retval
echo 0 > options/funcgraph-retval-hex
echo 1 > tracing_on
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
echo 0 > tracing_on
cat trace > ~/trace.log
To locate the root cause, search for error code -22 directly in the file
trace.log and identify the first function that returned -22. Once you
have identified this function, examine its code to determine the root
cause.
For example, in the trace log below, cpu_cgroup_can_attach
returned -22 first, so we can focus our analysis on this function to
identify the root cause.
...
1) | cgroup_migrate() {
1) 0.651 us | cgroup_migrate_add_task(); /* = 0xffff93fcfd346c00 */
1) | cgroup_migrate_execute() {
1) | cpu_cgroup_can_attach() {
1) | cgroup_taskset_first() {
1) 0.732 us | cgroup_taskset_next(); /* = 0xffff93fc8fb20000 */
1) 1.232 us | } /* cgroup_taskset_first = 0xffff93fc8fb20000 */
1) 0.380 us | sched_rt_can_attach(); /* = 0x0 */
1) 2.335 us | } /* cpu_cgroup_can_attach = -22 */
1) 4.369 us | } /* cgroup_migrate_execute = -22 */
1) 7.143 us | } /* cgroup_migrate = -22 */
...
Link: https://lkml.kernel.org/r/1fc502712c981e0e6742185ba242992170ac9da8.1680954589.git.pengdonglin@sangfor.com.cn
Tested-by: Florian Kauer <florian.kauer@linutronix.de>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Donglin Peng <pengdonglin@sangfor.com.cn>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-04-08 05:42:15 -07:00
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
function_graph: Support recording and printing the return value of function
Analyzing system call failures with the function_graph tracer can be a
time-consuming process, particularly when locating the kernel function
that first returns an error in the trace logs. This change aims to
simplify the process by recording the function return value to the
'retval' member of 'ftrace_graph_ret' and printing it when outputting
the trace log.
We have introduced new trace options: funcgraph-retval and
funcgraph-retval-hex. The former controls whether to display the return
value, while the latter controls the display format.
Please note that even if a function's return type is void, a return
value will still be printed. You can simply ignore it.
This patch only establishes the fundamental infrastructure. Subsequent
patches will make this feature available on some commonly used processor
architectures.
Here is an example:
I attempted to attach the demo process to a cpu cgroup, but it failed:
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
-bash: echo: write error: Invalid argument
The strace logs indicate that the write system call returned -EINVAL(-22):
...
write(1, "273\n", 4) = -1 EINVAL (Invalid argument)
...
To capture trace logs during a write system call, use the following
commands:
cd /sys/kernel/debug/tracing/
echo 0 > tracing_on
echo > trace
echo *sys_write > set_graph_function
echo *spin* > set_graph_notrace
echo *rcu* >> set_graph_notrace
echo *alloc* >> set_graph_notrace
echo preempt* >> set_graph_notrace
echo kfree* >> set_graph_notrace
echo $$ > set_ftrace_pid
echo function_graph > current_tracer
echo 1 > options/funcgraph-retval
echo 0 > options/funcgraph-retval-hex
echo 1 > tracing_on
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
echo 0 > tracing_on
cat trace > ~/trace.log
To locate the root cause, search for error code -22 directly in the file
trace.log and identify the first function that returned -22. Once you
have identified this function, examine its code to determine the root
cause.
For example, in the trace log below, cpu_cgroup_can_attach
returned -22 first, so we can focus our analysis on this function to
identify the root cause.
...
1) | cgroup_migrate() {
1) 0.651 us | cgroup_migrate_add_task(); /* = 0xffff93fcfd346c00 */
1) | cgroup_migrate_execute() {
1) | cpu_cgroup_can_attach() {
1) | cgroup_taskset_first() {
1) 0.732 us | cgroup_taskset_next(); /* = 0xffff93fc8fb20000 */
1) 1.232 us | } /* cgroup_taskset_first = 0xffff93fc8fb20000 */
1) 0.380 us | sched_rt_can_attach(); /* = 0x0 */
1) 2.335 us | } /* cpu_cgroup_can_attach = -22 */
1) 4.369 us | } /* cgroup_migrate_execute = -22 */
1) 7.143 us | } /* cgroup_migrate = -22 */
...
Link: https://lkml.kernel.org/r/1fc502712c981e0e6742185ba242992170ac9da8.1680954589.git.pengdonglin@sangfor.com.cn
Tested-by: Florian Kauer <florian.kauer@linutronix.de>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Donglin Peng <pengdonglin@sangfor.com.cn>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-04-08 05:42:15 -07:00
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-04-11 13:30:15 -04:00
|
|
|
|
function_graph: Support recording and printing the return value of function
Analyzing system call failures with the function_graph tracer can be a
time-consuming process, particularly when locating the kernel function
that first returns an error in the trace logs. This change aims to
simplify the process by recording the function return value to the
'retval' member of 'ftrace_graph_ret' and printing it when outputting
the trace log.
We have introduced new trace options: funcgraph-retval and
funcgraph-retval-hex. The former controls whether to display the return
value, while the latter controls the display format.
Please note that even if a function's return type is void, a return
value will still be printed. You can simply ignore it.
This patch only establishes the fundamental infrastructure. Subsequent
patches will make this feature available on some commonly used processor
architectures.
Here is an example:
I attempted to attach the demo process to a cpu cgroup, but it failed:
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
-bash: echo: write error: Invalid argument
The strace logs indicate that the write system call returned -EINVAL(-22):
...
write(1, "273\n", 4) = -1 EINVAL (Invalid argument)
...
To capture trace logs during a write system call, use the following
commands:
cd /sys/kernel/debug/tracing/
echo 0 > tracing_on
echo > trace
echo *sys_write > set_graph_function
echo *spin* > set_graph_notrace
echo *rcu* >> set_graph_notrace
echo *alloc* >> set_graph_notrace
echo preempt* >> set_graph_notrace
echo kfree* >> set_graph_notrace
echo $$ > set_ftrace_pid
echo function_graph > current_tracer
echo 1 > options/funcgraph-retval
echo 0 > options/funcgraph-retval-hex
echo 1 > tracing_on
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
echo 0 > tracing_on
cat trace > ~/trace.log
To locate the root cause, search for error code -22 directly in the file
trace.log and identify the first function that returned -22. Once you
have identified this function, examine its code to determine the root
cause.
For example, in the trace log below, cpu_cgroup_can_attach
returned -22 first, so we can focus our analysis on this function to
identify the root cause.
...
1) | cgroup_migrate() {
1) 0.651 us | cgroup_migrate_add_task(); /* = 0xffff93fcfd346c00 */
1) | cgroup_migrate_execute() {
1) | cpu_cgroup_can_attach() {
1) | cgroup_taskset_first() {
1) 0.732 us | cgroup_taskset_next(); /* = 0xffff93fc8fb20000 */
1) 1.232 us | } /* cgroup_taskset_first = 0xffff93fc8fb20000 */
1) 0.380 us | sched_rt_can_attach(); /* = 0x0 */
1) 2.335 us | } /* cpu_cgroup_can_attach = -22 */
1) 4.369 us | } /* cgroup_migrate_execute = -22 */
1) 7.143 us | } /* cgroup_migrate = -22 */
...
Link: https://lkml.kernel.org/r/1fc502712c981e0e6742185ba242992170ac9da8.1680954589.git.pengdonglin@sangfor.com.cn
Tested-by: Florian Kauer <florian.kauer@linutronix.de>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Donglin Peng <pengdonglin@sangfor.com.cn>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-04-08 05:42:15 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
|
|
|
|
|
function_graph: Support recording and printing the return value of function
Analyzing system call failures with the function_graph tracer can be a
time-consuming process, particularly when locating the kernel function
that first returns an error in the trace logs. This change aims to
simplify the process by recording the function return value to the
'retval' member of 'ftrace_graph_ret' and printing it when outputting
the trace log.
We have introduced new trace options: funcgraph-retval and
funcgraph-retval-hex. The former controls whether to display the return
value, while the latter controls the display format.
Please note that even if a function's return type is void, a return
value will still be printed. You can simply ignore it.
This patch only establishes the fundamental infrastructure. Subsequent
patches will make this feature available on some commonly used processor
architectures.
Here is an example:
I attempted to attach the demo process to a cpu cgroup, but it failed:
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
-bash: echo: write error: Invalid argument
The strace logs indicate that the write system call returned -EINVAL(-22):
...
write(1, "273\n", 4) = -1 EINVAL (Invalid argument)
...
To capture trace logs during a write system call, use the following
commands:
cd /sys/kernel/debug/tracing/
echo 0 > tracing_on
echo > trace
echo *sys_write > set_graph_function
echo *spin* > set_graph_notrace
echo *rcu* >> set_graph_notrace
echo *alloc* >> set_graph_notrace
echo preempt* >> set_graph_notrace
echo kfree* >> set_graph_notrace
echo $$ > set_ftrace_pid
echo function_graph > current_tracer
echo 1 > options/funcgraph-retval
echo 0 > options/funcgraph-retval-hex
echo 1 > tracing_on
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
echo 0 > tracing_on
cat trace > ~/trace.log
To locate the root cause, search for error code -22 directly in the file
trace.log and identify the first function that returned -22. Once you
have identified this function, examine its code to determine the root
cause.
For example, in the trace log below, cpu_cgroup_can_attach
returned -22 first, so we can focus our analysis on this function to
identify the root cause.
...
1) | cgroup_migrate() {
1) 0.651 us | cgroup_migrate_add_task(); /* = 0xffff93fcfd346c00 */
1) | cgroup_migrate_execute() {
1) | cpu_cgroup_can_attach() {
1) | cgroup_taskset_first() {
1) 0.732 us | cgroup_taskset_next(); /* = 0xffff93fc8fb20000 */
1) 1.232 us | } /* cgroup_taskset_first = 0xffff93fc8fb20000 */
1) 0.380 us | sched_rt_can_attach(); /* = 0x0 */
1) 2.335 us | } /* cpu_cgroup_can_attach = -22 */
1) 4.369 us | } /* cgroup_migrate_execute = -22 */
1) 7.143 us | } /* cgroup_migrate = -22 */
...
Link: https://lkml.kernel.org/r/1fc502712c981e0e6742185ba242992170ac9da8.1680954589.git.pengdonglin@sangfor.com.cn
Tested-by: Florian Kauer <florian.kauer@linutronix.de>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Donglin Peng <pengdonglin@sangfor.com.cn>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-04-08 05:42:15 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-02-06 18:30:44 +01:00
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
|
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
2015-09-30 09:42:05 -04:00
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
|
|
|
|
|
2018-01-31 23:48:49 +08:00
|
|
|
|
2008-11-28 00:42:46 +01:00
|
|
|
|
2008-11-26 00:57:25 +01:00
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
|
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
|
|
|
|
|
2025-01-21 19:44:36 -05:00
|
|
|
|
2008-11-26 00:16:27 -05:00
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
2010-02-26 17:08:16 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-02-26 17:08:16 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-12-08 20:54:49 -05:00
|
|
|
|
|
|
|
|
|
2010-02-26 17:08:16 -05:00
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
|
|
|
|
|
2011-06-03 16:58:48 +02:00
|
|
|
|
2015-09-30 09:42:05 -04:00
|
|
|
|
2008-11-28 00:42:46 +01:00
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
|
|
|
|
|
2008-11-26 00:57:25 +01:00
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
|
|
|
|
|
function_graph: Support recording and printing the return value of function
Analyzing system call failures with the function_graph tracer can be a
time-consuming process, particularly when locating the kernel function
that first returns an error in the trace logs. This change aims to
simplify the process by recording the function return value to the
'retval' member of 'ftrace_graph_ret' and printing it when outputting
the trace log.
We have introduced new trace options: funcgraph-retval and
funcgraph-retval-hex. The former controls whether to display the return
value, while the latter controls the display format.
Please note that even if a function's return type is void, a return
value will still be printed. You can simply ignore it.
This patch only establishes the fundamental infrastructure. Subsequent
patches will make this feature available on some commonly used processor
architectures.
Here is an example:
I attempted to attach the demo process to a cpu cgroup, but it failed:
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
-bash: echo: write error: Invalid argument
The strace logs indicate that the write system call returned -EINVAL(-22):
...
write(1, "273\n", 4) = -1 EINVAL (Invalid argument)
...
To capture trace logs during a write system call, use the following
commands:
cd /sys/kernel/debug/tracing/
echo 0 > tracing_on
echo > trace
echo *sys_write > set_graph_function
echo *spin* > set_graph_notrace
echo *rcu* >> set_graph_notrace
echo *alloc* >> set_graph_notrace
echo preempt* >> set_graph_notrace
echo kfree* >> set_graph_notrace
echo $$ > set_ftrace_pid
echo function_graph > current_tracer
echo 1 > options/funcgraph-retval
echo 0 > options/funcgraph-retval-hex
echo 1 > tracing_on
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
echo 0 > tracing_on
cat trace > ~/trace.log
To locate the root cause, search for error code -22 directly in the file
trace.log and identify the first function that returned -22. Once you
have identified this function, examine its code to determine the root
cause.
For example, in the trace log below, cpu_cgroup_can_attach
returned -22 first, so we can focus our analysis on this function to
identify the root cause.
...
1) | cgroup_migrate() {
1) 0.651 us | cgroup_migrate_add_task(); /* = 0xffff93fcfd346c00 */
1) | cgroup_migrate_execute() {
1) | cpu_cgroup_can_attach() {
1) | cgroup_taskset_first() {
1) 0.732 us | cgroup_taskset_next(); /* = 0xffff93fc8fb20000 */
1) 1.232 us | } /* cgroup_taskset_first = 0xffff93fc8fb20000 */
1) 0.380 us | sched_rt_can_attach(); /* = 0x0 */
1) 2.335 us | } /* cpu_cgroup_can_attach = -22 */
1) 4.369 us | } /* cgroup_migrate_execute = -22 */
1) 7.143 us | } /* cgroup_migrate = -22 */
...
Link: https://lkml.kernel.org/r/1fc502712c981e0e6742185ba242992170ac9da8.1680954589.git.pengdonglin@sangfor.com.cn
Tested-by: Florian Kauer <florian.kauer@linutronix.de>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Donglin Peng <pengdonglin@sangfor.com.cn>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-04-08 05:42:15 -07:00
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
function_graph: Support recording and printing the return value of function
Analyzing system call failures with the function_graph tracer can be a
time-consuming process, particularly when locating the kernel function
that first returns an error in the trace logs. This change aims to
simplify the process by recording the function return value to the
'retval' member of 'ftrace_graph_ret' and printing it when outputting
the trace log.
We have introduced new trace options: funcgraph-retval and
funcgraph-retval-hex. The former controls whether to display the return
value, while the latter controls the display format.
Please note that even if a function's return type is void, a return
value will still be printed. You can simply ignore it.
This patch only establishes the fundamental infrastructure. Subsequent
patches will make this feature available on some commonly used processor
architectures.
Here is an example:
I attempted to attach the demo process to a cpu cgroup, but it failed:
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
-bash: echo: write error: Invalid argument
The strace logs indicate that the write system call returned -EINVAL(-22):
...
write(1, "273\n", 4) = -1 EINVAL (Invalid argument)
...
To capture trace logs during a write system call, use the following
commands:
cd /sys/kernel/debug/tracing/
echo 0 > tracing_on
echo > trace
echo *sys_write > set_graph_function
echo *spin* > set_graph_notrace
echo *rcu* >> set_graph_notrace
echo *alloc* >> set_graph_notrace
echo preempt* >> set_graph_notrace
echo kfree* >> set_graph_notrace
echo $$ > set_ftrace_pid
echo function_graph > current_tracer
echo 1 > options/funcgraph-retval
echo 0 > options/funcgraph-retval-hex
echo 1 > tracing_on
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
echo 0 > tracing_on
cat trace > ~/trace.log
To locate the root cause, search for error code -22 directly in the file
trace.log and identify the first function that returned -22. Once you
have identified this function, examine its code to determine the root
cause.
For example, in the trace log below, cpu_cgroup_can_attach
returned -22 first, so we can focus our analysis on this function to
identify the root cause.
...
1) | cgroup_migrate() {
1) 0.651 us | cgroup_migrate_add_task(); /* = 0xffff93fcfd346c00 */
1) | cgroup_migrate_execute() {
1) | cpu_cgroup_can_attach() {
1) | cgroup_taskset_first() {
1) 0.732 us | cgroup_taskset_next(); /* = 0xffff93fc8fb20000 */
1) 1.232 us | } /* cgroup_taskset_first = 0xffff93fc8fb20000 */
1) 0.380 us | sched_rt_can_attach(); /* = 0x0 */
1) 2.335 us | } /* cpu_cgroup_can_attach = -22 */
1) 4.369 us | } /* cgroup_migrate_execute = -22 */
1) 7.143 us | } /* cgroup_migrate = -22 */
...
Link: https://lkml.kernel.org/r/1fc502712c981e0e6742185ba242992170ac9da8.1680954589.git.pengdonglin@sangfor.com.cn
Tested-by: Florian Kauer <florian.kauer@linutronix.de>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Donglin Peng <pengdonglin@sangfor.com.cn>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-04-08 05:42:15 -07:00
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-11-25 17:34:25 +08:00
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
2025-04-11 13:30:15 -04:00
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
2018-01-31 23:48:49 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2008-11-26 00:57:25 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
|
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
2008-11-26 00:57:25 +01:00
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
2015-09-30 09:42:05 -04:00
|
|
|
|
2024-08-13 13:11:06 -04:00
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-02-26 17:08:16 -05:00
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
|
|
|
|
|
2010-02-26 17:08:16 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-12-08 20:54:49 -05:00
|
|
|
|
|
|
|
|
|
2010-02-26 17:08:16 -05:00
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
2015-09-30 09:42:05 -04:00
|
|
|
|
2008-12-09 23:55:25 +01:00
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
|
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
2024-08-13 13:11:06 -04:00
|
|
|
|
|
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-11-25 17:34:25 +08:00
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
|
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
|
|
|
|
|
2009-02-06 18:30:44 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2009-03-19 11:29:23 -04:00
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
2015-09-30 09:42:05 -04:00
|
|
|
|
2009-03-19 11:29:23 -04:00
|
|
|
|
2008-11-26 00:57:25 +01:00
|
|
|
|
2008-11-28 00:42:46 +01:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2008-11-26 00:16:27 -05:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2009-03-19 11:29:23 -04:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
2025-10-31 11:46:11 +09:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2011-06-03 16:58:51 +02:00
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
|
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
2019-01-01 23:46:10 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-11-28 00:42:46 +01:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
|
|
|
|
|
2008-12-03 02:30:37 +01:00
|
|
|
|
|
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
|
|
|
|
|
2008-11-28 00:42:46 +01:00
|
|
|
|
2008-11-26 00:57:25 +01:00
|
|
|
|
2009-09-11 00:30:26 -04:00
|
|
|
|
2025-10-31 11:46:11 +09:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2009-09-11 00:30:26 -04:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2009-03-19 11:29:23 -04:00
|
|
|
|
|
|
|
|
|
2010-09-07 16:53:44 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-03-30 22:57:33 -03:00
|
|
|
|
2010-09-07 16:53:44 +02:00
|
|
|
|
2021-03-23 18:49:35 +01:00
|
|
|
|
2010-09-07 16:53:44 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-09-24 17:41:02 +02:00
|
|
|
|
2010-09-07 16:53:44 +02:00
|
|
|
|
|
|
|
|
|
2024-08-13 13:11:06 -04:00
|
|
|
|
|
|
|
|
|
2010-09-24 17:41:02 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-09-07 16:53:44 +02:00
|
|
|
|
|
|
|
|
|
2010-09-24 17:41:02 +02:00
|
|
|
|
|
|
|
|
|
2010-09-07 16:53:44 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-09-24 17:41:02 +02:00
|
|
|
|
2010-09-07 16:53:44 +02:00
|
|
|
|
|
|
|
|
|
2010-09-24 17:41:02 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-09-07 16:53:44 +02:00
|
|
|
|
|
|
|
|
|
2010-09-24 17:41:02 +02:00
|
|
|
|
|
|
|
|
|
2010-09-07 16:53:44 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-03-19 11:29:23 -04:00
|
|
|
|
|
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
2009-03-19 11:29:23 -04:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
2009-03-19 11:29:23 -04:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
|
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-11-25 17:34:25 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
|
|
|
|
|
2025-11-25 17:34:25 +08:00
|
|
|
|
|
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-03-19 11:29:23 -04:00
|
|
|
|
2010-09-07 16:53:44 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2009-03-19 11:29:23 -04:00
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
2009-02-06 18:30:44 +01:00
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-01-21 19:44:36 -05:00
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
|
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
2025-01-21 19:44:36 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
2015-09-30 09:42:05 -04:00
|
|
|
|
2024-08-13 13:11:06 -04:00
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
|
|
|
|
|
2010-02-26 17:08:16 -05:00
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
|
|
|
|
|
2024-08-13 13:11:06 -04:00
|
|
|
|
|
|
|
|
|
2010-09-07 16:53:44 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
2010-02-26 17:08:16 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-02-26 17:08:16 -05:00
|
|
|
|
|
|
|
|
|
2016-12-08 20:54:49 -05:00
|
|
|
|
|
|
|
|
|
2010-02-26 17:08:16 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
2011-06-03 16:58:48 +02:00
|
|
|
|
2015-09-30 09:42:05 -04:00
|
|
|
|
2008-11-28 00:42:46 +01:00
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
|
|
|
|
|
2008-11-26 00:57:25 +01:00
|
|
|
|
2010-02-26 17:08:16 -05:00
|
|
|
|
function_graph: Support recording and printing the return value of function
Analyzing system call failures with the function_graph tracer can be a
time-consuming process, particularly when locating the kernel function
that first returns an error in the trace logs. This change aims to
simplify the process by recording the function return value to the
'retval' member of 'ftrace_graph_ret' and printing it when outputting
the trace log.
We have introduced new trace options: funcgraph-retval and
funcgraph-retval-hex. The former controls whether to display the return
value, while the latter controls the display format.
Please note that even if a function's return type is void, a return
value will still be printed. You can simply ignore it.
This patch only establishes the fundamental infrastructure. Subsequent
patches will make this feature available on some commonly used processor
architectures.
Here is an example:
I attempted to attach the demo process to a cpu cgroup, but it failed:
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
-bash: echo: write error: Invalid argument
The strace logs indicate that the write system call returned -EINVAL(-22):
...
write(1, "273\n", 4) = -1 EINVAL (Invalid argument)
...
To capture trace logs during a write system call, use the following
commands:
cd /sys/kernel/debug/tracing/
echo 0 > tracing_on
echo > trace
echo *sys_write > set_graph_function
echo *spin* > set_graph_notrace
echo *rcu* >> set_graph_notrace
echo *alloc* >> set_graph_notrace
echo preempt* >> set_graph_notrace
echo kfree* >> set_graph_notrace
echo $$ > set_ftrace_pid
echo function_graph > current_tracer
echo 1 > options/funcgraph-retval
echo 0 > options/funcgraph-retval-hex
echo 1 > tracing_on
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
echo 0 > tracing_on
cat trace > ~/trace.log
To locate the root cause, search for error code -22 directly in the file
trace.log and identify the first function that returned -22. Once you
have identified this function, examine its code to determine the root
cause.
For example, in the trace log below, cpu_cgroup_can_attach
returned -22 first, so we can focus our analysis on this function to
identify the root cause.
...
1) | cgroup_migrate() {
1) 0.651 us | cgroup_migrate_add_task(); /* = 0xffff93fcfd346c00 */
1) | cgroup_migrate_execute() {
1) | cpu_cgroup_can_attach() {
1) | cgroup_taskset_first() {
1) 0.732 us | cgroup_taskset_next(); /* = 0xffff93fc8fb20000 */
1) 1.232 us | } /* cgroup_taskset_first = 0xffff93fc8fb20000 */
1) 0.380 us | sched_rt_can_attach(); /* = 0x0 */
1) 2.335 us | } /* cpu_cgroup_can_attach = -22 */
1) 4.369 us | } /* cgroup_migrate_execute = -22 */
1) 7.143 us | } /* cgroup_migrate = -22 */
...
Link: https://lkml.kernel.org/r/1fc502712c981e0e6742185ba242992170ac9da8.1680954589.git.pengdonglin@sangfor.com.cn
Tested-by: Florian Kauer <florian.kauer@linutronix.de>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Donglin Peng <pengdonglin@sangfor.com.cn>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-04-08 05:42:15 -07:00
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
2010-02-26 17:08:16 -05:00
|
|
|
|
function_graph: Support recording and printing the return value of function
Analyzing system call failures with the function_graph tracer can be a
time-consuming process, particularly when locating the kernel function
that first returns an error in the trace logs. This change aims to
simplify the process by recording the function return value to the
'retval' member of 'ftrace_graph_ret' and printing it when outputting
the trace log.
We have introduced new trace options: funcgraph-retval and
funcgraph-retval-hex. The former controls whether to display the return
value, while the latter controls the display format.
Please note that even if a function's return type is void, a return
value will still be printed. You can simply ignore it.
This patch only establishes the fundamental infrastructure. Subsequent
patches will make this feature available on some commonly used processor
architectures.
Here is an example:
I attempted to attach the demo process to a cpu cgroup, but it failed:
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
-bash: echo: write error: Invalid argument
The strace logs indicate that the write system call returned -EINVAL(-22):
...
write(1, "273\n", 4) = -1 EINVAL (Invalid argument)
...
To capture trace logs during a write system call, use the following
commands:
cd /sys/kernel/debug/tracing/
echo 0 > tracing_on
echo > trace
echo *sys_write > set_graph_function
echo *spin* > set_graph_notrace
echo *rcu* >> set_graph_notrace
echo *alloc* >> set_graph_notrace
echo preempt* >> set_graph_notrace
echo kfree* >> set_graph_notrace
echo $$ > set_ftrace_pid
echo function_graph > current_tracer
echo 1 > options/funcgraph-retval
echo 0 > options/funcgraph-retval-hex
echo 1 > tracing_on
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
echo 0 > tracing_on
cat trace > ~/trace.log
To locate the root cause, search for error code -22 directly in the file
trace.log and identify the first function that returned -22. Once you
have identified this function, examine its code to determine the root
cause.
For example, in the trace log below, cpu_cgroup_can_attach
returned -22 first, so we can focus our analysis on this function to
identify the root cause.
...
1) | cgroup_migrate() {
1) 0.651 us | cgroup_migrate_add_task(); /* = 0xffff93fcfd346c00 */
1) | cgroup_migrate_execute() {
1) | cpu_cgroup_can_attach() {
1) | cgroup_taskset_first() {
1) 0.732 us | cgroup_taskset_next(); /* = 0xffff93fc8fb20000 */
1) 1.232 us | } /* cgroup_taskset_first = 0xffff93fc8fb20000 */
1) 0.380 us | sched_rt_can_attach(); /* = 0x0 */
1) 2.335 us | } /* cpu_cgroup_can_attach = -22 */
1) 4.369 us | } /* cgroup_migrate_execute = -22 */
1) 7.143 us | } /* cgroup_migrate = -22 */
...
Link: https://lkml.kernel.org/r/1fc502712c981e0e6742185ba242992170ac9da8.1680954589.git.pengdonglin@sangfor.com.cn
Tested-by: Florian Kauer <florian.kauer@linutronix.de>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Donglin Peng <pengdonglin@sangfor.com.cn>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-04-08 05:42:15 -07:00
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
|
|
|
|
|
function_graph: Support recording and printing the return value of function
Analyzing system call failures with the function_graph tracer can be a
time-consuming process, particularly when locating the kernel function
that first returns an error in the trace logs. This change aims to
simplify the process by recording the function return value to the
'retval' member of 'ftrace_graph_ret' and printing it when outputting
the trace log.
We have introduced new trace options: funcgraph-retval and
funcgraph-retval-hex. The former controls whether to display the return
value, while the latter controls the display format.
Please note that even if a function's return type is void, a return
value will still be printed. You can simply ignore it.
This patch only establishes the fundamental infrastructure. Subsequent
patches will make this feature available on some commonly used processor
architectures.
Here is an example:
I attempted to attach the demo process to a cpu cgroup, but it failed:
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
-bash: echo: write error: Invalid argument
The strace logs indicate that the write system call returned -EINVAL(-22):
...
write(1, "273\n", 4) = -1 EINVAL (Invalid argument)
...
To capture trace logs during a write system call, use the following
commands:
cd /sys/kernel/debug/tracing/
echo 0 > tracing_on
echo > trace
echo *sys_write > set_graph_function
echo *spin* > set_graph_notrace
echo *rcu* >> set_graph_notrace
echo *alloc* >> set_graph_notrace
echo preempt* >> set_graph_notrace
echo kfree* >> set_graph_notrace
echo $$ > set_ftrace_pid
echo function_graph > current_tracer
echo 1 > options/funcgraph-retval
echo 0 > options/funcgraph-retval-hex
echo 1 > tracing_on
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
echo 0 > tracing_on
cat trace > ~/trace.log
To locate the root cause, search for error code -22 directly in the file
trace.log and identify the first function that returned -22. Once you
have identified this function, examine its code to determine the root
cause.
For example, in the trace log below, cpu_cgroup_can_attach
returned -22 first, so we can focus our analysis on this function to
identify the root cause.
...
1) | cgroup_migrate() {
1) 0.651 us | cgroup_migrate_add_task(); /* = 0xffff93fcfd346c00 */
1) | cgroup_migrate_execute() {
1) | cpu_cgroup_can_attach() {
1) | cgroup_taskset_first() {
1) 0.732 us | cgroup_taskset_next(); /* = 0xffff93fc8fb20000 */
1) 1.232 us | } /* cgroup_taskset_first = 0xffff93fc8fb20000 */
1) 0.380 us | sched_rt_can_attach(); /* = 0x0 */
1) 2.335 us | } /* cpu_cgroup_can_attach = -22 */
1) 4.369 us | } /* cgroup_migrate_execute = -22 */
1) 7.143 us | } /* cgroup_migrate = -22 */
...
Link: https://lkml.kernel.org/r/1fc502712c981e0e6742185ba242992170ac9da8.1680954589.git.pengdonglin@sangfor.com.cn
Tested-by: Florian Kauer <florian.kauer@linutronix.de>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Donglin Peng <pengdonglin@sangfor.com.cn>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-04-08 05:42:15 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-04-11 13:30:15 -04:00
|
|
|
|
function_graph: Support recording and printing the return value of function
Analyzing system call failures with the function_graph tracer can be a
time-consuming process, particularly when locating the kernel function
that first returns an error in the trace logs. This change aims to
simplify the process by recording the function return value to the
'retval' member of 'ftrace_graph_ret' and printing it when outputting
the trace log.
We have introduced new trace options: funcgraph-retval and
funcgraph-retval-hex. The former controls whether to display the return
value, while the latter controls the display format.
Please note that even if a function's return type is void, a return
value will still be printed. You can simply ignore it.
This patch only establishes the fundamental infrastructure. Subsequent
patches will make this feature available on some commonly used processor
architectures.
Here is an example:
I attempted to attach the demo process to a cpu cgroup, but it failed:
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
-bash: echo: write error: Invalid argument
The strace logs indicate that the write system call returned -EINVAL(-22):
...
write(1, "273\n", 4) = -1 EINVAL (Invalid argument)
...
To capture trace logs during a write system call, use the following
commands:
cd /sys/kernel/debug/tracing/
echo 0 > tracing_on
echo > trace
echo *sys_write > set_graph_function
echo *spin* > set_graph_notrace
echo *rcu* >> set_graph_notrace
echo *alloc* >> set_graph_notrace
echo preempt* >> set_graph_notrace
echo kfree* >> set_graph_notrace
echo $$ > set_ftrace_pid
echo function_graph > current_tracer
echo 1 > options/funcgraph-retval
echo 0 > options/funcgraph-retval-hex
echo 1 > tracing_on
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
echo 0 > tracing_on
cat trace > ~/trace.log
To locate the root cause, search for error code -22 directly in the file
trace.log and identify the first function that returned -22. Once you
have identified this function, examine its code to determine the root
cause.
For example, in the trace log below, cpu_cgroup_can_attach
returned -22 first, so we can focus our analysis on this function to
identify the root cause.
...
1) | cgroup_migrate() {
1) 0.651 us | cgroup_migrate_add_task(); /* = 0xffff93fcfd346c00 */
1) | cgroup_migrate_execute() {
1) | cpu_cgroup_can_attach() {
1) | cgroup_taskset_first() {
1) 0.732 us | cgroup_taskset_next(); /* = 0xffff93fc8fb20000 */
1) 1.232 us | } /* cgroup_taskset_first = 0xffff93fc8fb20000 */
1) 0.380 us | sched_rt_can_attach(); /* = 0x0 */
1) 2.335 us | } /* cpu_cgroup_can_attach = -22 */
1) 4.369 us | } /* cgroup_migrate_execute = -22 */
1) 7.143 us | } /* cgroup_migrate = -22 */
...
Link: https://lkml.kernel.org/r/1fc502712c981e0e6742185ba242992170ac9da8.1680954589.git.pengdonglin@sangfor.com.cn
Tested-by: Florian Kauer <florian.kauer@linutronix.de>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Donglin Peng <pengdonglin@sangfor.com.cn>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-04-08 05:42:15 -07:00
|
|
|
|
2025-04-11 13:30:15 -04:00
|
|
|
|
function_graph: Support recording and printing the return value of function
Analyzing system call failures with the function_graph tracer can be a
time-consuming process, particularly when locating the kernel function
that first returns an error in the trace logs. This change aims to
simplify the process by recording the function return value to the
'retval' member of 'ftrace_graph_ret' and printing it when outputting
the trace log.
We have introduced new trace options: funcgraph-retval and
funcgraph-retval-hex. The former controls whether to display the return
value, while the latter controls the display format.
Please note that even if a function's return type is void, a return
value will still be printed. You can simply ignore it.
This patch only establishes the fundamental infrastructure. Subsequent
patches will make this feature available on some commonly used processor
architectures.
Here is an example:
I attempted to attach the demo process to a cpu cgroup, but it failed:
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
-bash: echo: write error: Invalid argument
The strace logs indicate that the write system call returned -EINVAL(-22):
...
write(1, "273\n", 4) = -1 EINVAL (Invalid argument)
...
To capture trace logs during a write system call, use the following
commands:
cd /sys/kernel/debug/tracing/
echo 0 > tracing_on
echo > trace
echo *sys_write > set_graph_function
echo *spin* > set_graph_notrace
echo *rcu* >> set_graph_notrace
echo *alloc* >> set_graph_notrace
echo preempt* >> set_graph_notrace
echo kfree* >> set_graph_notrace
echo $$ > set_ftrace_pid
echo function_graph > current_tracer
echo 1 > options/funcgraph-retval
echo 0 > options/funcgraph-retval-hex
echo 1 > tracing_on
echo `pidof demo` > /sys/fs/cgroup/cpu/test/tasks
echo 0 > tracing_on
cat trace > ~/trace.log
To locate the root cause, search for error code -22 directly in the file
trace.log and identify the first function that returned -22. Once you
have identified this function, examine its code to determine the root
cause.
For example, in the trace log below, cpu_cgroup_can_attach
returned -22 first, so we can focus our analysis on this function to
identify the root cause.
...
1) | cgroup_migrate() {
1) 0.651 us | cgroup_migrate_add_task(); /* = 0xffff93fcfd346c00 */
1) | cgroup_migrate_execute() {
1) | cpu_cgroup_can_attach() {
1) | cgroup_taskset_first() {
1) 0.732 us | cgroup_taskset_next(); /* = 0xffff93fc8fb20000 */
1) 1.232 us | } /* cgroup_taskset_first = 0xffff93fc8fb20000 */
1) 0.380 us | sched_rt_can_attach(); /* = 0x0 */
1) 2.335 us | } /* cpu_cgroup_can_attach = -22 */
1) 4.369 us | } /* cgroup_migrate_execute = -22 */
1) 7.143 us | } /* cgroup_migrate = -22 */
...
Link: https://lkml.kernel.org/r/1fc502712c981e0e6742185ba242992170ac9da8.1680954589.git.pengdonglin@sangfor.com.cn
Tested-by: Florian Kauer <florian.kauer@linutronix.de>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Donglin Peng <pengdonglin@sangfor.com.cn>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2023-04-08 05:42:15 -07:00
|
|
|
|
2025-04-11 13:30:15 -04:00
|
|
|
|
2008-11-27 01:46:33 +01:00
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2020-10-28 08:19:24 -04:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2008-12-09 23:55:25 +01:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
|
|
|
|
|
2008-12-09 23:55:25 +01:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2008-11-26 00:57:25 +01:00
|
|
|
|
2008-11-25 21:07:04 +01:00
|
|
|
|
2008-12-03 23:45:11 +01:00
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
|
|
|
|
|
2008-12-03 23:45:11 +01:00
|
|
|
|
2015-09-30 09:42:05 -04:00
|
|
|
|
|
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
2009-03-19 15:14:46 -04:00
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
2008-12-03 23:45:11 +01:00
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2009-02-18 04:25:25 +01:00
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
2015-09-30 09:42:05 -04:00
|
|
|
|
2008-12-03 23:45:11 +01:00
|
|
|
|
|
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
|
|
|
|
|
2008-12-03 23:45:11 +01:00
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2009-03-06 17:21:49 +01:00
|
|
|
|
2009-03-19 15:14:46 -04:00
|
|
|
|
2016-09-01 11:43:54 +09:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-03-19 15:14:46 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-04-22 18:46:14 -04:00
|
|
|
|
2009-03-19 15:14:46 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-12-03 23:45:11 +01:00
|
|
|
|
2014-11-14 15:49:41 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-12-24 01:43:25 +01:00
|
|
|
|
2014-06-25 15:54:42 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-12-24 01:43:25 +01:00
|
|
|
|
|
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2014-11-14 15:49:41 -05:00
|
|
|
|
2014-11-12 14:57:38 -05:00
|
|
|
|
2008-12-03 23:45:11 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-11-25 21:07:04 +01:00
|
|
|
|
2011-06-03 16:58:47 +02:00
|
|
|
|
2008-11-25 21:07:04 +01:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
|
|
|
|
|
2008-11-25 21:07:04 +01:00
|
|
|
|
2009-03-19 15:14:46 -04:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-11-25 21:07:04 +01:00
|
|
|
|
2008-11-26 00:57:25 +01:00
|
|
|
|
|
|
|
|
|
2008-11-25 21:07:04 +01:00
|
|
|
|
2025-02-27 13:58:06 -05:00
|
|
|
|
2008-11-26 00:57:25 +01:00
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
2025-11-25 17:34:25 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-11-25 17:34:25 +08:00
|
|
|
|
|
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
2008-11-26 00:57:25 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-01-21 19:44:36 -05:00
|
|
|
|
2008-11-26 00:57:25 +01:00
|
|
|
|
2010-04-02 19:01:22 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-11-26 00:57:25 +01:00
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
2008-11-25 21:07:04 +01:00
|
|
|
|
2009-03-19 15:14:46 -04:00
|
|
|
|
|
|
|
|
|
2008-11-25 21:07:04 +01:00
|
|
|
|
|
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-11-11 18:24:09 -05:00
|
|
|
|
|
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
|
|
|
|
|
2010-04-02 19:01:20 +02:00
|
|
|
|
2010-04-22 18:46:14 -04:00
|
|
|
|
|
|
|
|
|
2010-04-02 19:01:20 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
2009-09-11 00:30:26 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
2009-09-11 00:30:26 -04:00
|
|
|
|
2019-01-01 23:46:10 +08:00
|
|
|
|
|
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
2009-09-11 00:30:26 -04:00
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
2009-09-11 00:30:26 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-06-03 16:58:50 +02:00
|
|
|
|
2009-09-11 00:30:26 -04:00
|
|
|
|
|
|
|
|
|
2015-09-30 09:42:05 -04:00
|
|
|
|
|
|
|
|
|
2008-12-08 01:56:06 +01:00
|
|
|
|
2025-10-31 11:46:11 +09:00
|
|
|
|
2009-09-11 00:30:26 -04:00
|
|
|
|
|
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
2009-09-11 00:30:26 -04:00
|
|
|
|
2008-12-08 01:56:06 +01:00
|
|
|
|
2014-11-08 21:42:12 +01:00
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
2014-11-08 21:42:10 +01:00
|
|
|
|
2019-01-01 23:46:10 +08:00
|
|
|
|
|
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
2014-11-08 21:42:10 +01:00
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
2014-11-08 21:42:10 +01:00
|
|
|
|
2009-09-11 00:30:26 -04:00
|
|
|
|
2019-01-01 23:46:12 +08:00
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
2014-11-08 21:42:10 +01:00
|
|
|
|
|
|
|
|
|
2008-12-08 01:56:06 +01:00
|
|
|
|
|
|
|
|
|
2014-11-08 21:42:12 +01:00
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
2014-11-08 21:42:10 +01:00
|
|
|
|
2019-01-01 23:46:10 +08:00
|
|
|
|
|
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
2014-11-08 21:42:10 +01:00
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
2014-11-08 21:42:10 +01:00
|
|
|
|
2009-09-11 00:30:26 -04:00
|
|
|
|
2019-01-01 23:46:12 +08:00
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
2014-11-08 21:42:10 +01:00
|
|
|
|
|
|
|
|
|
2008-12-08 01:56:06 +01:00
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
2014-07-18 18:07:49 -04:00
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
2025-11-11 18:24:09 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-04-02 19:01:21 +02:00
|
|
|
|
|
|
|
|
|
2010-09-23 14:00:52 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-09-30 09:42:05 -04:00
|
|
|
|
2010-09-23 14:00:52 +02:00
|
|
|
|
2025-10-31 11:46:11 +09:00
|
|
|
|
2011-06-03 16:58:51 +02:00
|
|
|
|
|
|
|
|
|
2025-10-31 11:46:11 +09:00
|
|
|
|
2010-09-23 14:00:52 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-06-03 16:58:47 +02:00
|
|
|
|
2010-09-23 14:00:52 +02:00
|
|
|
|
2015-09-30 09:42:05 -04:00
|
|
|
|
2010-09-23 14:00:52 +02:00
|
|
|
|
|
|
|
|
|
2010-04-02 19:01:22 +02:00
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
2015-04-13 22:30:12 +02:00
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
|
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
|
|
|
|
|
2015-04-13 22:30:12 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
|
|
|
|
|
2015-04-13 22:30:12 +02:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-09-07 16:53:44 +02:00
|
|
|
|
|
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-09-07 16:53:44 +02:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
2009-03-19 13:24:42 -04:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-03-22 14:28:09 -07:00
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
|
|
|
|
|
2010-04-02 19:01:22 +02:00
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
tracing: Fix use-after-free in print_graph_function_flags during tracer switching
Kairui reported a UAF issue in print_graph_function_flags() during
ftrace stress testing [1]. This issue can be reproduced if puting a
'mdelay(10)' after 'mutex_unlock(&trace_types_lock)' in s_start(),
and executing the following script:
$ echo function_graph > current_tracer
$ cat trace > /dev/null &
$ sleep 5 # Ensure the 'cat' reaches the 'mdelay(10)' point
$ echo timerlat > current_tracer
The root cause lies in the two calls to print_graph_function_flags
within print_trace_line during each s_show():
* One through 'iter->trace->print_line()';
* Another through 'event->funcs->trace()', which is hidden in
print_trace_fmt() before print_trace_line returns.
Tracer switching only updates the former, while the latter continues
to use the print_line function of the old tracer, which in the script
above is print_graph_function_flags.
Moreover, when switching from the 'function_graph' tracer to the
'timerlat' tracer, s_start only calls graph_trace_close of the
'function_graph' tracer to free 'iter->private', but does not set
it to NULL. This provides an opportunity for 'event->funcs->trace()'
to use an invalid 'iter->private'.
To fix this issue, set 'iter->private' to NULL immediately after
freeing it in graph_trace_close(), ensuring that an invalid pointer
is not passed to other tracers. Additionally, clean up the unnecessary
'iter->private = NULL' during each 'cat trace' when using wakeup and
irqsoff tracers.
[1] https://lore.kernel.org/all/20231112150030.84609-1-ryncsn@gmail.com/
Cc: stable@vger.kernel.org
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Zheng Yejian <zhengyejian1@huawei.com>
Link: https://lore.kernel.org/20250320122137.23635-1-wutengda@huaweicloud.com
Fixes: eecb91b9f98d ("tracing: Fix memleak due to race between current_tracer and trace")
Closes: https://lore.kernel.org/all/CAMgjq7BW79KDSCyp+tZHjShSzHsScSiJxn5ffskp-QzVM06fxw@mail.gmail.com/
Reported-by: Kairui Song <kasong@tencent.com>
Signed-off-by: Tengda Wu <wutengda@huaweicloud.com>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2025-03-20 12:21:37 +00:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
|
|
|
|
|
2014-01-10 11:13:54 -05:00
|
|
|
|
|
|
|
|
|
2010-09-14 18:58:33 -04:00
|
|
|
|
2025-11-14 14:22:32 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-09-29 19:06:50 -04:00
|
|
|
|
2025-11-14 14:22:30 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-11-14 14:22:33 -05:00
|
|
|
|
|
|
|
|
|
2025-11-14 14:22:32 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-11-14 14:22:33 -05:00
|
|
|
|
2025-11-14 14:22:32 -05:00
|
|
|
|
2025-11-14 14:22:33 -05:00
|
|
|
|
2025-11-14 14:22:30 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-11-14 14:22:33 -05:00
|
|
|
|
2025-11-14 14:22:30 -05:00
|
|
|
|
2025-11-14 14:22:33 -05:00
|
|
|
|
2025-02-27 13:58:07 -05:00
|
|
|
|
2025-11-14 14:22:33 -05:00
|
|
|
|
2025-02-27 13:58:07 -05:00
|
|
|
|
2010-09-14 18:58:33 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-04-22 18:46:14 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-04-02 19:01:20 +02:00
|
|
|
|
|
|
|
|
|
2010-04-22 18:46:14 -04:00
|
|
|
|
2010-04-02 19:01:20 +02:00
|
|
|
|
|
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-04-02 19:01:20 +02:00
|
|
|
|
|
|
|
|
|
2010-04-22 18:46:14 -04:00
|
|
|
|
2010-04-02 19:01:20 +02:00
|
|
|
|
|
|
|
|
|
2013-07-18 14:41:51 -04:00
|
|
|
|
2009-03-10 14:10:56 -04:00
|
|
|
|
2014-07-18 15:17:27 +04:00
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
2009-01-22 17:04:53 -08:00
|
|
|
|
2009-11-24 13:57:38 +01:00
|
|
|
|
2009-03-10 14:10:56 -04:00
|
|
|
|
|
|
|
|
|
2008-12-08 01:56:06 +01:00
|
|
|
|
|
|
|
|
|
2025-11-11 18:24:09 -05:00
|
|
|
|
2010-09-14 18:58:33 -04:00
|
|
|
|
2024-06-03 15:07:12 -04:00
|
|
|
|
2009-02-07 21:33:57 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-11-25 21:07:04 +01:00
|
|
|
|
|
|
|
|
|
2013-01-16 10:49:37 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-12-08 19:28:28 -05:00
|
|
|
|
2013-01-16 10:49:37 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-12-08 19:28:28 -05:00
|
|
|
|
2013-01-16 10:49:37 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-20 12:13:40 -05:00
|
|
|
|
2013-01-16 10:49:37 -05:00
|
|
|
|
2020-07-12 09:10:36 +08:00
|
|
|
|
2013-01-16 10:49:37 -05:00
|
|
|
|
2020-07-12 09:10:36 +08:00
|
|
|
|
|
|
|
|
|
2013-01-16 10:49:37 -05:00
|
|
|
|
|
|
|
|
|
2021-08-18 11:24:51 -04:00
|
|
|
|
2013-01-16 10:49:37 -05:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-20 12:13:40 -05:00
|
|
|
|
2013-01-16 10:49:37 -05:00
|
|
|
|
2008-11-25 21:07:04 +01:00
|
|
|
|
|
|
|
|
|
2017-09-08 16:14:18 -07:00
|
|
|
|
2009-07-28 20:26:06 +08:00
|
|
|
|
2015-05-05 09:39:12 -04:00
|
|
|
|
2016-03-22 14:28:09 -07:00
|
|
|
|
2010-04-02 19:01:20 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-09-14 20:29:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-05-05 09:39:12 -04:00
|
|
|
|
2016-03-22 14:28:09 -07:00
|
|
|
|
2010-04-02 19:01:20 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2008-11-25 21:07:04 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-10-05 12:13:07 -04:00
|
|
|
|