linux/tools/perf/builtin-trace.c

3192 lines
86 KiB

perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Support 'strace' syscall event groups I.e.: $ cat ~/share/perf-core/strace/groups/file access chmod creat execve faccessat getcwd lstat mkdir open openat quotactl readlink rename rmdir stat statfs symlink unlink $ Then, on a quiet desktop, try running this and then moving your mouse to see the deluge of mouse related activity: # perf probe 'vfs_getname=getname_flags:72 pathname=filename:string' Added new event: probe:vfs_getname (on getname_flags:72 with pathname=filename:string) You can now use it in all perf tools, such as: perf record -e probe:vfs_getname -aR sleep 1 # # trace --ev probe:vfs_getname --filter-pids 2232 -e file 0.042 (0.042 ms): mousetweaks/2235 open(filename: 0x14e3910, mode: 438 ) ... 0.042 ( ): probe:vfs_getname:(ffffffff812230bc) pathname="/home/acme/.icons/Adwaita/cursors/xterm") 0.100 (0.100 ms): mousetweaks/2235 ... [continued]: open()) = -1 ENOENT No such file or directory 0.142 (0.018 ms): mousetweaks/2235 open(filename: 0x14c3c10, mode: 438 ) ... 0.142 ( ): probe:vfs_getname:(ffffffff812230bc) pathname="/home/acme/.icons/Adwaita/index.theme") 0.192 (0.069 ms): mousetweaks/2235 ... [continued]: open()) = -1 ENOENT No such file or directory 0.230 (0.017 ms): mousetweaks/2235 open(filename: 0x14c3c10, mode: 438 ) ... 0.230 ( ): probe:vfs_getname:(ffffffff812230bc) pathname="/usr/share/icons/Adwaita/cursors/xterm") 0.253 (0.041 ms): mousetweaks/2235 ... [continued]: open()) = 14 0.459 (0.008 ms): mousetweaks/2235 open(filename: 0x14e3910, mode: 438 ) ... 0.459 ( ): probe:vfs_getname:(ffffffff812230bc) pathname="/home/acme/.icons/Adwaita/cursors/left_side") 0.468 (0.017 ms): mousetweaks/2235 ... [continued]: open()) = -1 ENOENT No such file or directory Need to combine that raw_syscalls:sys_enter(open) + probe:vfs_getname + raw_syscalls:sys_exit(open) sequence... Now, if you're bored, please write some more syscall groups, like the ones in 'strace' and send it our way :-) Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Milian Wolff <mail@milianw.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-a42xklu59lcbxp7bbnic74a8@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-07-20 12:02:09 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Add summary option to dump syscall statistics When enabled dumps a summary of all syscalls by task with the usual statistics -- min, max, average and relative stddev. For example, make - 26341 : 3344 [ 17.4% ] 0.000 ms read : 52 0.000 4.802 0.644 30.08 write : 20 0.004 0.036 0.010 21.72 open : 24 0.003 0.046 0.014 23.68 close : 64 0.002 0.055 0.008 22.53 stat : 2714 0.002 0.222 0.004 4.47 fstat : 18 0.001 0.041 0.006 46.26 mmap : 30 0.003 0.009 0.006 5.71 mprotect : 8 0.006 0.039 0.016 32.16 munmap : 12 0.007 0.077 0.020 38.25 brk : 48 0.002 0.014 0.004 10.18 rt_sigaction : 18 0.002 0.002 0.002 2.11 rt_sigprocmask : 60 0.002 0.128 0.010 32.88 access : 2 0.006 0.006 0.006 0.00 pipe : 12 0.004 0.048 0.013 35.98 vfork : 34 0.448 0.980 0.692 3.04 execve : 20 0.000 0.387 0.046 56.66 wait4 : 34 0.017 9923.287 593.221 68.45 fcntl : 8 0.001 0.041 0.013 48.79 getdents : 48 0.002 0.079 0.013 19.62 getcwd : 2 0.005 0.005 0.005 0.00 chdir : 2 0.070 0.070 0.070 0.00 getrlimit : 2 0.045 0.045 0.045 0.00 arch_prctl : 2 0.002 0.002 0.002 0.00 setrlimit : 2 0.002 0.002 0.002 0.00 openat : 94 0.003 0.005 0.003 2.11 Signed-off-by: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung.kim@lge.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1381289214-24885-3-git-send-email-dsahern@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-08 21:26:53 -06:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Add beautifier for mmap prot parm [root@zoo ~]# perf trace -e mmap,mprotect sleep 1 0.984 ( 0.015 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: 34, fd: 4294967295, off: 0) = 0xd62ae000 1.114 ( 0.016 ms): mmap(addr: 0, len: 125100, prot: READ, flags: 2, fd: 3, off: 0 ) = 0xd628f000 1.252 ( 0.020 ms): mmap(addr: 0x33c1600000, len: 3896312, prot: EXEC|READ, flags: 2050, fd: 3, off: 0) = 0xc1600000 1.282 ( 0.024 ms): mprotect(start: 0x33c17ad000, len: 2097152, prot: NONE ) = 0 1.315 ( 0.026 ms): mmap(addr: 0x33c19ad000, len: 24576, prot: READ|WRITE, flags: 2066, fd: 3, off: 1757184) = 0xc19ad000 1.352 ( 0.017 ms): mmap(addr: 0x33c19b3000, len: 17400, prot: READ|WRITE, flags: 50, fd: 4294967295, off: 0) = 0xc19b3000 1.415 ( 0.011 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: 34, fd: 4294967295, off: 0) = 0xd628e000 1.440 ( 0.011 ms): mmap(addr: 0, len: 8192, prot: READ|WRITE, flags: 34, fd: 4294967295, off: 0) = 0xd628c000 1.569 ( 0.019 ms): mprotect(start: 0x606000, len: 4096, prot: READ ) = 0 1.591 ( 0.017 ms): mprotect(start: 0x33c19ad000, len: 16384, prot: READ ) = 0 1.616 ( 0.016 ms): mprotect(start: 0x33c1420000, len: 4096, prot: READ ) = 0 2.105 ( 0.018 ms): mmap(addr: 0, len: 104789808, prot: READ, flags: 2, fd: 3, off: 0 ) = 0xcfe9c000 [root@zoo ~]# Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-q1ubhdd9wigxneam616ggdsn@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-08-20 17:44:42 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
trace: Beautify perf_event_open syscall Syswide tracing and then running 'stat' and 'trace': $ perf trace -e perf_event_open 1034.649 (0.019 ms): perf/6133 perf_event_open(attr_uptr: 0x36f0360, pid: 16134, cpu: -1, group_fd: -1, flags: FD_CLOEXEC) = -1 EINVAL Invalid argument 1034.670 (0.008 ms): perf/6133 perf_event_open(attr_uptr: 0x36f0360, pid: 16134, cpu: -1, group_fd: -1) = -1 EINVAL Invalid argument 1034.681 (0.007 ms): perf/6133 perf_event_open(attr_uptr: 0x36f0360, pid: 16134, cpu: -1, group_fd: -1) = -1 EINVAL Invalid argument 1034.692 (0.007 ms): perf/6133 perf_event_open(attr_uptr: 0x36f0360, pid: 16134, cpu: -1, group_fd: -1) = -1 EINVAL Invalid argument 9986.983 (0.014 ms): trace/6139 perf_event_open(attr_uptr: 0x7ffd9c629320, pid: -1, group_fd: -1, flags: FD_CLOEXEC) = 3 9987.026 (0.016 ms): trace/6139 perf_event_open(attr_uptr: 0x37c7e70, pid: -1, group_fd: -1, flags: FD_CLOEXEC) = 3 9987.041 (0.008 ms): trace/6139 perf_event_open(attr_uptr: 0x37c7e70, pid: -1, group_fd: -1, flags: FD_CLOEXEC) = 3 9987.489 (0.092 ms): trace/6139 perf_event_open(attr_uptr: 0x3795ee0, pid: 16140, group_fd: -1, flags: FD_CLOEXEC) = 3 9987.536 (0.044 ms): trace/6139 perf_event_open(attr_uptr: 0x3795ee0, pid: 16140, cpu: 1, group_fd: -1, flags: FD_CLOEXEC) = 4 9987.580 (0.041 ms): trace/6139 perf_event_open(attr_uptr: 0x3795ee0, pid: 16140, cpu: 2, group_fd: -1, flags: FD_CLOEXEC) = 5 9987.620 (0.037 ms): trace/6139 perf_event_open(attr_uptr: 0x3795ee0, pid: 16140, cpu: 3, group_fd: -1, flags: FD_CLOEXEC) = 7 9987.659 (0.035 ms): trace/6139 perf_event_open(attr_uptr: 0x37975d0, pid: 16140, group_fd: -1, flags: FD_CLOEXEC) = 8 9987.692 (0.031 ms): trace/6139 perf_event_open(attr_uptr: 0x37975d0, pid: 16140, cpu: 1, group_fd: -1, flags: FD_CLOEXEC) = 9 9987.727 (0.032 ms): trace/6139 perf_event_open(attr_uptr: 0x37975d0, pid: 16140, cpu: 2, group_fd: -1, flags: FD_CLOEXEC) = 10 9987.761 (0.031 ms): trace/6139 perf_event_open(attr_uptr: 0x37975d0, pid: 16140, cpu: 3, group_fd: -1, flags: FD_CLOEXEC) = 11 Need to intercept perf_copy_attr() with a kprobe or with eBPF... Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Link: http://lkml.kernel.org/n/tip-njb105hab2i3t5dexym9lskl@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-06-11 22:47:54 -03:00
perf trace: Fix SIGBUS failures due to misaligned accesses On Sparc64 perf-trace is failing in many spots due to extended load instructions being used on misaligned accesses. (gdb) run trace ls Starting program: /tmp/perf/perf trace ls [Thread debugging using libthread_db enabled] Detaching after fork from child process 169460. <ls output removed> Program received signal SIGBUS, Bus error. 0x000000000014f4dc in tp_field__u64 (field=0x4cc700, sample=0x7feffffa098) at builtin-trace.c:61 warning: Source file is more recent than executable. 61 TP_UINT_FIELD(64); (gdb) bt 0 0x000000000014f4dc in tp_field__u64 (field=0x4cc700, sample=0x7feffffa098) at builtin-trace.c:61 1 0x0000000000156ad4 in trace__sys_exit (trace=0x7feffffc268, evsel=0x4cc580, event=0xfffffc0104912000, sample=0x7feffffa098) at builtin-trace.c:1701 2 0x0000000000158c14 in trace__run (trace=0x7feffffc268, argc=1, argv=0x7fefffff360) at builtin-trace.c:2160 3 0x000000000015b78c in cmd_trace (argc=1, argv=0x7fefffff360, prefix=0x0) at builtin-trace.c:2609 4 0x0000000000107d94 in run_builtin (p=0x4549c8, argc=2, argv=0x7fefffff360) at perf.c:341 5 0x0000000000108140 in handle_internal_command (argc=2, argv=0x7fefffff360) at perf.c:400 6 0x0000000000108308 in run_argv (argcp=0x7feffffef2c, argv=0x7feffffef20) at perf.c:444 7 0x0000000000108728 in main (argc=2, argv=0x7fefffff360) at perf.c:559 (gdb) p *sample $1 = {ip = 4391276, pid = 169472, tid = 169472, time = 6303014583281250, addr = 0, id = 72082, stream_id = 18446744073709551615, period = 1, weight = 0, transaction = 0, cpu = 73, raw_size = 36, data_src = 84410401, flags = 0, insn_len = 0, raw_data = 0xfffffc010491203c, callchain = 0x0, branch_stack = 0x0, user_regs = {abi = 0, mask = 0, regs = 0x0, cache_regs = 0x7feffffa098, cache_mask = 0}, intr_regs = {abi = 0, mask = 0, regs = 0x0, cache_regs = 0x7feffffa098, cache_mask = 0}, user_stack = { offset = 0, size = 0, data = 0x0}, read = {time_enabled = 0, time_running = 0, {group = {nr = 0, values = 0x0}, one = {value = 0, id = 0}}}} (gdb) p *field $2 = {offset = 16, {integer = 0x14f4a8 <tp_field__u64>, pointer = 0x14f4a8 <tp_field__u64>}} sample->raw_data is guaranteed to not be 8-byte aligned because it is preceded by the size as a u3. So accessing raw data with an extended load instruction causes the SIGBUS. Resolve by using memcpy to a temporary variable of appropriate size. Signed-off-by: David Ahern <david.ahern@oracle.com> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1424376022-140608-1-git-send-email-david.ahern@oracle.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-19 15:00:22 -05:00
perf trace: Fix SIGBUS failures due to misaligned accesses On Sparc64 perf-trace is failing in many spots due to extended load instructions being used on misaligned accesses. (gdb) run trace ls Starting program: /tmp/perf/perf trace ls [Thread debugging using libthread_db enabled] Detaching after fork from child process 169460. <ls output removed> Program received signal SIGBUS, Bus error. 0x000000000014f4dc in tp_field__u64 (field=0x4cc700, sample=0x7feffffa098) at builtin-trace.c:61 warning: Source file is more recent than executable. 61 TP_UINT_FIELD(64); (gdb) bt 0 0x000000000014f4dc in tp_field__u64 (field=0x4cc700, sample=0x7feffffa098) at builtin-trace.c:61 1 0x0000000000156ad4 in trace__sys_exit (trace=0x7feffffc268, evsel=0x4cc580, event=0xfffffc0104912000, sample=0x7feffffa098) at builtin-trace.c:1701 2 0x0000000000158c14 in trace__run (trace=0x7feffffc268, argc=1, argv=0x7fefffff360) at builtin-trace.c:2160 3 0x000000000015b78c in cmd_trace (argc=1, argv=0x7fefffff360, prefix=0x0) at builtin-trace.c:2609 4 0x0000000000107d94 in run_builtin (p=0x4549c8, argc=2, argv=0x7fefffff360) at perf.c:341 5 0x0000000000108140 in handle_internal_command (argc=2, argv=0x7fefffff360) at perf.c:400 6 0x0000000000108308 in run_argv (argcp=0x7feffffef2c, argv=0x7feffffef20) at perf.c:444 7 0x0000000000108728 in main (argc=2, argv=0x7fefffff360) at perf.c:559 (gdb) p *sample $1 = {ip = 4391276, pid = 169472, tid = 169472, time = 6303014583281250, addr = 0, id = 72082, stream_id = 18446744073709551615, period = 1, weight = 0, transaction = 0, cpu = 73, raw_size = 36, data_src = 84410401, flags = 0, insn_len = 0, raw_data = 0xfffffc010491203c, callchain = 0x0, branch_stack = 0x0, user_regs = {abi = 0, mask = 0, regs = 0x0, cache_regs = 0x7feffffa098, cache_mask = 0}, intr_regs = {abi = 0, mask = 0, regs = 0x0, cache_regs = 0x7feffffa098, cache_mask = 0}, user_stack = { offset = 0, size = 0, data = 0x0}, read = {time_enabled = 0, time_running = 0, {group = {nr = 0, values = 0x0}, one = {value = 0, id = 0}}}} (gdb) p *field $2 = {offset = 16, {integer = 0x14f4a8 <tp_field__u64>, pointer = 0x14f4a8 <tp_field__u64>}} sample->raw_data is guaranteed to not be 8-byte aligned because it is preceded by the size as a u3. So accessing raw data with an extended load instruction causes the SIGBUS. Resolve by using memcpy to a temporary variable of appropriate size. Signed-off-by: David Ahern <david.ahern@oracle.com> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1424376022-140608-1-git-send-email-david.ahern@oracle.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-19 15:00:22 -05:00
perf trace: Initial beautifier for ioctl's 'cmd' arg [root@zoo linux]# trace -e ioctl | grep -v "cmd: 0x" | head -10 0.386 ( 0.001 ms): trace/1602 ioctl(fd: 1<pipe:[127057]>, cmd: TCGETS, arg: 0x7fff59fcb4d0 ) = -1 ENOTTY Inappropriate ioctl for device 1459.368 ( 0.002 ms): inotify_reader/10352 ioctl(fd: 18<anon_inode:inotify>, cmd: FIONREAD, arg: 0x7fb835228bcc ) = 0 1463.586 ( 0.002 ms): inotify_reader/10352 ioctl(fd: 18<anon_inode:inotify>, cmd: FIONREAD, arg: 0x7fb835228bcc ) = 0 1463.611 ( 0.002 ms): inotify_reader/10352 ioctl(fd: 18<anon_inode:inotify>, cmd: FIONREAD, arg: 0x7fb835228bcc ) = 0 3740.526 ( 0.002 ms): awk/1612 ioctl(fd: 1<pipe:[128265]>, cmd: TCGETS, arg: 0x7fff4d166b90 ) = -1 ENOTTY Inappropriate ioctl for device 3740.704 ( 0.001 ms): awk/1612 ioctl(fd: 3</proc/meminfo>, cmd: TCGETS, arg: 0x7fff4d1669a0 ) = -1 ENOTTY Inappropriate ioctl for device 3742.550 ( 0.002 ms): ps/1614 ioctl(fd: 1<pipe:[128266]>, cmd: TIOCGWINSZ, arg: 0x7fff591762b0 ) = -1 ENOTTY Inappropriate ioctl for device 3742.555 ( 0.003 ms): ps/1614 ioctl(fd: 2<socket:[19550]>, cmd: TIOCGWINSZ, arg: 0x7fff591762b0 ) = -1 ENOTTY Inappropriate ioctl for device 3742.558 ( 0.002 ms): ps/1614 ioctl(cmd: TIOCGWINSZ, arg: 0x7fff591762b0 ) = -1 ENOTTY Inappropriate ioctl for device 3742.572 ( 0.002 ms): ps/1614 ioctl(fd: 1<pipe:[128266]>, cmd: TCGETS, arg: 0x7fff59176220 ) = -1 ENOTTY Inappropriate ioctl for device [root@zoo linux]# Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-afajwap3mr60dfl4qpdl1pxn@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-08 17:43:20 -03:00
perf trace: Initial beautifier for ioctl's 'cmd' arg [root@zoo linux]# trace -e ioctl | grep -v "cmd: 0x" | head -10 0.386 ( 0.001 ms): trace/1602 ioctl(fd: 1<pipe:[127057]>, cmd: TCGETS, arg: 0x7fff59fcb4d0 ) = -1 ENOTTY Inappropriate ioctl for device 1459.368 ( 0.002 ms): inotify_reader/10352 ioctl(fd: 18<anon_inode:inotify>, cmd: FIONREAD, arg: 0x7fb835228bcc ) = 0 1463.586 ( 0.002 ms): inotify_reader/10352 ioctl(fd: 18<anon_inode:inotify>, cmd: FIONREAD, arg: 0x7fb835228bcc ) = 0 1463.611 ( 0.002 ms): inotify_reader/10352 ioctl(fd: 18<anon_inode:inotify>, cmd: FIONREAD, arg: 0x7fb835228bcc ) = 0 3740.526 ( 0.002 ms): awk/1612 ioctl(fd: 1<pipe:[128265]>, cmd: TCGETS, arg: 0x7fff4d166b90 ) = -1 ENOTTY Inappropriate ioctl for device 3740.704 ( 0.001 ms): awk/1612 ioctl(fd: 3</proc/meminfo>, cmd: TCGETS, arg: 0x7fff4d1669a0 ) = -1 ENOTTY Inappropriate ioctl for device 3742.550 ( 0.002 ms): ps/1614 ioctl(fd: 1<pipe:[128266]>, cmd: TIOCGWINSZ, arg: 0x7fff591762b0 ) = -1 ENOTTY Inappropriate ioctl for device 3742.555 ( 0.003 ms): ps/1614 ioctl(fd: 2<socket:[19550]>, cmd: TIOCGWINSZ, arg: 0x7fff591762b0 ) = -1 ENOTTY Inappropriate ioctl for device 3742.558 ( 0.002 ms): ps/1614 ioctl(cmd: TIOCGWINSZ, arg: 0x7fff591762b0 ) = -1 ENOTTY Inappropriate ioctl for device 3742.572 ( 0.002 ms): ps/1614 ioctl(fd: 1<pipe:[128266]>, cmd: TCGETS, arg: 0x7fff59176220 ) = -1 ENOTTY Inappropriate ioctl for device [root@zoo linux]# Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-afajwap3mr60dfl4qpdl1pxn@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-08 17:43:20 -03:00
trace: Beautify perf_event_open syscall Syswide tracing and then running 'stat' and 'trace': $ perf trace -e perf_event_open 1034.649 (0.019 ms): perf/6133 perf_event_open(attr_uptr: 0x36f0360, pid: 16134, cpu: -1, group_fd: -1, flags: FD_CLOEXEC) = -1 EINVAL Invalid argument 1034.670 (0.008 ms): perf/6133 perf_event_open(attr_uptr: 0x36f0360, pid: 16134, cpu: -1, group_fd: -1) = -1 EINVAL Invalid argument 1034.681 (0.007 ms): perf/6133 perf_event_open(attr_uptr: 0x36f0360, pid: 16134, cpu: -1, group_fd: -1) = -1 EINVAL Invalid argument 1034.692 (0.007 ms): perf/6133 perf_event_open(attr_uptr: 0x36f0360, pid: 16134, cpu: -1, group_fd: -1) = -1 EINVAL Invalid argument 9986.983 (0.014 ms): trace/6139 perf_event_open(attr_uptr: 0x7ffd9c629320, pid: -1, group_fd: -1, flags: FD_CLOEXEC) = 3 9987.026 (0.016 ms): trace/6139 perf_event_open(attr_uptr: 0x37c7e70, pid: -1, group_fd: -1, flags: FD_CLOEXEC) = 3 9987.041 (0.008 ms): trace/6139 perf_event_open(attr_uptr: 0x37c7e70, pid: -1, group_fd: -1, flags: FD_CLOEXEC) = 3 9987.489 (0.092 ms): trace/6139 perf_event_open(attr_uptr: 0x3795ee0, pid: 16140, group_fd: -1, flags: FD_CLOEXEC) = 3 9987.536 (0.044 ms): trace/6139 perf_event_open(attr_uptr: 0x3795ee0, pid: 16140, cpu: 1, group_fd: -1, flags: FD_CLOEXEC) = 4 9987.580 (0.041 ms): trace/6139 perf_event_open(attr_uptr: 0x3795ee0, pid: 16140, cpu: 2, group_fd: -1, flags: FD_CLOEXEC) = 5 9987.620 (0.037 ms): trace/6139 perf_event_open(attr_uptr: 0x3795ee0, pid: 16140, cpu: 3, group_fd: -1, flags: FD_CLOEXEC) = 7 9987.659 (0.035 ms): trace/6139 perf_event_open(attr_uptr: 0x37975d0, pid: 16140, group_fd: -1, flags: FD_CLOEXEC) = 8 9987.692 (0.031 ms): trace/6139 perf_event_open(attr_uptr: 0x37975d0, pid: 16140, cpu: 1, group_fd: -1, flags: FD_CLOEXEC) = 9 9987.727 (0.032 ms): trace/6139 perf_event_open(attr_uptr: 0x37975d0, pid: 16140, cpu: 2, group_fd: -1, flags: FD_CLOEXEC) = 10 9987.761 (0.031 ms): trace/6139 perf_event_open(attr_uptr: 0x37975d0, pid: 16140, cpu: 3, group_fd: -1, flags: FD_CLOEXEC) = 11 Need to intercept perf_copy_attr() with a kprobe or with eBPF... Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Link: http://lkml.kernel.org/n/tip-njb105hab2i3t5dexym9lskl@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-06-11 22:47:54 -03:00
perf trace: Add beautifier for mmap prot parm [root@zoo ~]# perf trace -e mmap,mprotect sleep 1 0.984 ( 0.015 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: 34, fd: 4294967295, off: 0) = 0xd62ae000 1.114 ( 0.016 ms): mmap(addr: 0, len: 125100, prot: READ, flags: 2, fd: 3, off: 0 ) = 0xd628f000 1.252 ( 0.020 ms): mmap(addr: 0x33c1600000, len: 3896312, prot: EXEC|READ, flags: 2050, fd: 3, off: 0) = 0xc1600000 1.282 ( 0.024 ms): mprotect(start: 0x33c17ad000, len: 2097152, prot: NONE ) = 0 1.315 ( 0.026 ms): mmap(addr: 0x33c19ad000, len: 24576, prot: READ|WRITE, flags: 2066, fd: 3, off: 1757184) = 0xc19ad000 1.352 ( 0.017 ms): mmap(addr: 0x33c19b3000, len: 17400, prot: READ|WRITE, flags: 50, fd: 4294967295, off: 0) = 0xc19b3000 1.415 ( 0.011 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: 34, fd: 4294967295, off: 0) = 0xd628e000 1.440 ( 0.011 ms): mmap(addr: 0, len: 8192, prot: READ|WRITE, flags: 34, fd: 4294967295, off: 0) = 0xd628c000 1.569 ( 0.019 ms): mprotect(start: 0x606000, len: 4096, prot: READ ) = 0 1.591 ( 0.017 ms): mprotect(start: 0x33c19ad000, len: 16384, prot: READ ) = 0 1.616 ( 0.016 ms): mprotect(start: 0x33c1420000, len: 4096, prot: READ ) = 0 2.105 ( 0.018 ms): mmap(addr: 0, len: 104789808, prot: READ, flags: 2, fd: 3, off: 0 ) = 0xcfe9c000 [root@zoo ~]# Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-q1ubhdd9wigxneam616ggdsn@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-08-20 17:44:42 -03:00
perf trace: Add beautifier for mmap prot parm [root@zoo ~]# perf trace -e mmap,mprotect sleep 1 0.984 ( 0.015 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: 34, fd: 4294967295, off: 0) = 0xd62ae000 1.114 ( 0.016 ms): mmap(addr: 0, len: 125100, prot: READ, flags: 2, fd: 3, off: 0 ) = 0xd628f000 1.252 ( 0.020 ms): mmap(addr: 0x33c1600000, len: 3896312, prot: EXEC|READ, flags: 2050, fd: 3, off: 0) = 0xc1600000 1.282 ( 0.024 ms): mprotect(start: 0x33c17ad000, len: 2097152, prot: NONE ) = 0 1.315 ( 0.026 ms): mmap(addr: 0x33c19ad000, len: 24576, prot: READ|WRITE, flags: 2066, fd: 3, off: 1757184) = 0xc19ad000 1.352 ( 0.017 ms): mmap(addr: 0x33c19b3000, len: 17400, prot: READ|WRITE, flags: 50, fd: 4294967295, off: 0) = 0xc19b3000 1.415 ( 0.011 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: 34, fd: 4294967295, off: 0) = 0xd628e000 1.440 ( 0.011 ms): mmap(addr: 0, len: 8192, prot: READ|WRITE, flags: 34, fd: 4294967295, off: 0) = 0xd628c000 1.569 ( 0.019 ms): mprotect(start: 0x606000, len: 4096, prot: READ ) = 0 1.591 ( 0.017 ms): mprotect(start: 0x33c19ad000, len: 16384, prot: READ ) = 0 1.616 ( 0.016 ms): mprotect(start: 0x33c1420000, len: 4096, prot: READ ) = 0 2.105 ( 0.018 ms): mmap(addr: 0, len: 104789808, prot: READ, flags: 2, fd: 3, off: 0 ) = 0xcfe9c000 [root@zoo ~]# Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-q1ubhdd9wigxneam616ggdsn@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-08-20 17:44:42 -03:00
perf trace: Add beautifier for mmap flags parm [root@zoo ~]# perf trace -e mmap,mprotect sleep 1 0.992 ( 0.015 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: 4294967295, off: 0) = 0xa60be000 1.108 ( 0.012 ms): mmap(addr: 0, len: 125100, prot: READ, flags: PRIVATE, fd: 3, off: 0 ) = 0xa609f000 1.209 ( 0.014 ms): mmap(addr: 0x33c1600000, len: 3896312, prot: EXEC|READ, flags: PRIVATE|DENYWRITE, fd: 3, off: 0) = 0xc1600000 1.232 ( 0.018 ms): mprotect(start: 0x33c17ad000, len: 2097152, prot: NONE ) = 0 1.255 ( 0.018 ms): mmap(addr: 0x33c19ad000, len: 24576, prot: READ|WRITE, flags: PRIVATE|DENYWRITE|FIXED, fd: 3, off: 1757184) = 0xc19ad000 1.281 ( 0.011 ms): mmap(addr: 0x33c19b3000, len: 17400, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS|FIXED, fd: 4294967295, off: 0) = 0xc19b3000 1.328 ( 0.008 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: 4294967295, off: 0) = 0xa609e000 1.346 ( 0.008 ms): mmap(addr: 0, len: 8192, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: 4294967295, off: 0) = 0xa609c000 1.443 ( 0.013 ms): mprotect(start: 0x606000, len: 4096, prot: READ ) = 0 1.459 ( 0.011 ms): mprotect(start: 0x33c19ad000, len: 16384, prot: READ ) = 0 1.477 ( 0.011 ms): mprotect(start: 0x33c1420000, len: 4096, prot: READ ) = 0 1.855 ( 0.013 ms): mmap(addr: 0, len: 104789808, prot: READ, flags: PRIVATE, fd: 3, off: 0) = 0x9fcac000 [root@zoo ~]# Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-q1ubhdd9wigxneam616ggdsn@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-08-23 10:48:33 -03:00
perf trace: Add beautifier for mmap flags parm [root@zoo ~]# perf trace -e mmap,mprotect sleep 1 0.992 ( 0.015 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: 4294967295, off: 0) = 0xa60be000 1.108 ( 0.012 ms): mmap(addr: 0, len: 125100, prot: READ, flags: PRIVATE, fd: 3, off: 0 ) = 0xa609f000 1.209 ( 0.014 ms): mmap(addr: 0x33c1600000, len: 3896312, prot: EXEC|READ, flags: PRIVATE|DENYWRITE, fd: 3, off: 0) = 0xc1600000 1.232 ( 0.018 ms): mprotect(start: 0x33c17ad000, len: 2097152, prot: NONE ) = 0 1.255 ( 0.018 ms): mmap(addr: 0x33c19ad000, len: 24576, prot: READ|WRITE, flags: PRIVATE|DENYWRITE|FIXED, fd: 3, off: 1757184) = 0xc19ad000 1.281 ( 0.011 ms): mmap(addr: 0x33c19b3000, len: 17400, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS|FIXED, fd: 4294967295, off: 0) = 0xc19b3000 1.328 ( 0.008 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: 4294967295, off: 0) = 0xa609e000 1.346 ( 0.008 ms): mmap(addr: 0, len: 8192, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: 4294967295, off: 0) = 0xa609c000 1.443 ( 0.013 ms): mprotect(start: 0x606000, len: 4096, prot: READ ) = 0 1.459 ( 0.011 ms): mprotect(start: 0x33c19ad000, len: 16384, prot: READ ) = 0 1.477 ( 0.011 ms): mprotect(start: 0x33c1420000, len: 4096, prot: READ ) = 0 1.855 ( 0.013 ms): mmap(addr: 0, len: 104789808, prot: READ, flags: PRIVATE, fd: 3, off: 0) = 0x9fcac000 [root@zoo ~]# Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-q1ubhdd9wigxneam616ggdsn@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-08-23 10:48:33 -03:00
perf trace: Add beautifier for mmap flags parm [root@zoo ~]# perf trace -e mmap,mprotect sleep 1 0.992 ( 0.015 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: 4294967295, off: 0) = 0xa60be000 1.108 ( 0.012 ms): mmap(addr: 0, len: 125100, prot: READ, flags: PRIVATE, fd: 3, off: 0 ) = 0xa609f000 1.209 ( 0.014 ms): mmap(addr: 0x33c1600000, len: 3896312, prot: EXEC|READ, flags: PRIVATE|DENYWRITE, fd: 3, off: 0) = 0xc1600000 1.232 ( 0.018 ms): mprotect(start: 0x33c17ad000, len: 2097152, prot: NONE ) = 0 1.255 ( 0.018 ms): mmap(addr: 0x33c19ad000, len: 24576, prot: READ|WRITE, flags: PRIVATE|DENYWRITE|FIXED, fd: 3, off: 1757184) = 0xc19ad000 1.281 ( 0.011 ms): mmap(addr: 0x33c19b3000, len: 17400, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS|FIXED, fd: 4294967295, off: 0) = 0xc19b3000 1.328 ( 0.008 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: 4294967295, off: 0) = 0xa609e000 1.346 ( 0.008 ms): mmap(addr: 0, len: 8192, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: 4294967295, off: 0) = 0xa609c000 1.443 ( 0.013 ms): mprotect(start: 0x606000, len: 4096, prot: READ ) = 0 1.459 ( 0.011 ms): mprotect(start: 0x33c19ad000, len: 16384, prot: READ ) = 0 1.477 ( 0.011 ms): mprotect(start: 0x33c1420000, len: 4096, prot: READ ) = 0 1.855 ( 0.013 ms): mmap(addr: 0, len: 104789808, prot: READ, flags: PRIVATE, fd: 3, off: 0) = 0x9fcac000 [root@zoo ~]# Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-q1ubhdd9wigxneam616ggdsn@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-08-23 10:48:33 -03:00
perf trace: Add beautifier for mmap flags parm [root@zoo ~]# perf trace -e mmap,mprotect sleep 1 0.992 ( 0.015 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: 4294967295, off: 0) = 0xa60be000 1.108 ( 0.012 ms): mmap(addr: 0, len: 125100, prot: READ, flags: PRIVATE, fd: 3, off: 0 ) = 0xa609f000 1.209 ( 0.014 ms): mmap(addr: 0x33c1600000, len: 3896312, prot: EXEC|READ, flags: PRIVATE|DENYWRITE, fd: 3, off: 0) = 0xc1600000 1.232 ( 0.018 ms): mprotect(start: 0x33c17ad000, len: 2097152, prot: NONE ) = 0 1.255 ( 0.018 ms): mmap(addr: 0x33c19ad000, len: 24576, prot: READ|WRITE, flags: PRIVATE|DENYWRITE|FIXED, fd: 3, off: 1757184) = 0xc19ad000 1.281 ( 0.011 ms): mmap(addr: 0x33c19b3000, len: 17400, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS|FIXED, fd: 4294967295, off: 0) = 0xc19b3000 1.328 ( 0.008 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: 4294967295, off: 0) = 0xa609e000 1.346 ( 0.008 ms): mmap(addr: 0, len: 8192, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: 4294967295, off: 0) = 0xa609c000 1.443 ( 0.013 ms): mprotect(start: 0x606000, len: 4096, prot: READ ) = 0 1.459 ( 0.011 ms): mprotect(start: 0x33c19ad000, len: 16384, prot: READ ) = 0 1.477 ( 0.011 ms): mprotect(start: 0x33c1420000, len: 4096, prot: READ ) = 0 1.855 ( 0.013 ms): mmap(addr: 0, len: 104789808, prot: READ, flags: PRIVATE, fd: 3, off: 0) = 0x9fcac000 [root@zoo ~]# Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-q1ubhdd9wigxneam616ggdsn@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-08-23 10:48:33 -03:00
perf trace: Add beautifier for mmap flags parm [root@zoo ~]# perf trace -e mmap,mprotect sleep 1 0.992 ( 0.015 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: 4294967295, off: 0) = 0xa60be000 1.108 ( 0.012 ms): mmap(addr: 0, len: 125100, prot: READ, flags: PRIVATE, fd: 3, off: 0 ) = 0xa609f000 1.209 ( 0.014 ms): mmap(addr: 0x33c1600000, len: 3896312, prot: EXEC|READ, flags: PRIVATE|DENYWRITE, fd: 3, off: 0) = 0xc1600000 1.232 ( 0.018 ms): mprotect(start: 0x33c17ad000, len: 2097152, prot: NONE ) = 0 1.255 ( 0.018 ms): mmap(addr: 0x33c19ad000, len: 24576, prot: READ|WRITE, flags: PRIVATE|DENYWRITE|FIXED, fd: 3, off: 1757184) = 0xc19ad000 1.281 ( 0.011 ms): mmap(addr: 0x33c19b3000, len: 17400, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS|FIXED, fd: 4294967295, off: 0) = 0xc19b3000 1.328 ( 0.008 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: 4294967295, off: 0) = 0xa609e000 1.346 ( 0.008 ms): mmap(addr: 0, len: 8192, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: 4294967295, off: 0) = 0xa609c000 1.443 ( 0.013 ms): mprotect(start: 0x606000, len: 4096, prot: READ ) = 0 1.459 ( 0.011 ms): mprotect(start: 0x33c19ad000, len: 16384, prot: READ ) = 0 1.477 ( 0.011 ms): mprotect(start: 0x33c1420000, len: 4096, prot: READ ) = 0 1.855 ( 0.013 ms): mmap(addr: 0, len: 104789808, prot: READ, flags: PRIVATE, fd: 3, off: 0) = 0x9fcac000 [root@zoo ~]# Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-q1ubhdd9wigxneam616ggdsn@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-08-23 10:48:33 -03:00
perf trace: Add beautifier for mmap flags parm [root@zoo ~]# perf trace -e mmap,mprotect sleep 1 0.992 ( 0.015 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: 4294967295, off: 0) = 0xa60be000 1.108 ( 0.012 ms): mmap(addr: 0, len: 125100, prot: READ, flags: PRIVATE, fd: 3, off: 0 ) = 0xa609f000 1.209 ( 0.014 ms): mmap(addr: 0x33c1600000, len: 3896312, prot: EXEC|READ, flags: PRIVATE|DENYWRITE, fd: 3, off: 0) = 0xc1600000 1.232 ( 0.018 ms): mprotect(start: 0x33c17ad000, len: 2097152, prot: NONE ) = 0 1.255 ( 0.018 ms): mmap(addr: 0x33c19ad000, len: 24576, prot: READ|WRITE, flags: PRIVATE|DENYWRITE|FIXED, fd: 3, off: 1757184) = 0xc19ad000 1.281 ( 0.011 ms): mmap(addr: 0x33c19b3000, len: 17400, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS|FIXED, fd: 4294967295, off: 0) = 0xc19b3000 1.328 ( 0.008 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: 4294967295, off: 0) = 0xa609e000 1.346 ( 0.008 ms): mmap(addr: 0, len: 8192, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: 4294967295, off: 0) = 0xa609c000 1.443 ( 0.013 ms): mprotect(start: 0x606000, len: 4096, prot: READ ) = 0 1.459 ( 0.011 ms): mprotect(start: 0x33c19ad000, len: 16384, prot: READ ) = 0 1.477 ( 0.011 ms): mprotect(start: 0x33c1420000, len: 4096, prot: READ ) = 0 1.855 ( 0.013 ms): mmap(addr: 0, len: 104789808, prot: READ, flags: PRIVATE, fd: 3, off: 0) = 0x9fcac000 [root@zoo ~]# Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-q1ubhdd9wigxneam616ggdsn@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-08-23 10:48:33 -03:00
perf trace: Beautify keyctl's option arg 8.697 (0.103 ms): pool/2343 keyctl(option: GET_PERSISTENT, arg2: 1000, arg3: 4294967294, arg4: 140703061514067, arg5: 140703692383680) = 1023192809 8.763 (0.049 ms): pool/2343 keyctl(option: SEARCH, arg2: 1023192809, arg3: 140703745767772, arg4: 140703745767832, arg5: 4294967294) = 140224497 8.789 (0.016 ms): pool/2343 keyctl(option: SEARCH, arg2: 140224497, arg3: 140703745767814, arg4: 140703745767900) = 512300257 8.807 (0.011 ms): pool/2343 keyctl(option: READ, arg2: 512300257 ) = 13 8.822 (0.008 ms): pool/2343 keyctl(option: READ, arg2: 512300257, arg3: 140703061514000, arg4: 13 ) = 13 8.837 (0.007 ms): pool/2343 keyctl(option: READ, arg2: 140224497 ) = 4 8.852 (0.009 ms): pool/2343 keyctl(option: READ, arg2: 140224497, arg3: 140703061514000, arg4: 4 ) = 4 8.869 (0.010 ms): pool/2343 keyctl(option: SEARCH, arg2: 140224497, arg3: 140703745767772, arg4: 140703061514032) = -1 ENOKEY Required key not available 8.892 (0.017 ms): pool/2343 keyctl(option: DESCRIBE, arg2: 512300257 ) = 43 8.910 (0.012 ms): pool/2343 keyctl(option: DESCRIBE, arg2: 512300257, arg3: 140703061544384, arg4: 43) = 43 Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-013ab219irsxngyumrf5gp8s@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-08-11 11:05:36 -03:00
perf trace: Deref sys_enter pointer args with contents from probe:vfs_getname To work like strace and dereference syscall pointer args we need to insert probes (or tracepoints) right after we copy those bytes from userspace. Since we're formatting the syscall args at raw_syscalls:sys_enter time, we need to have a formatter that just stores the position where, later, when we get the probe:vfs_getname, we can insert the pointer contents. Now, if a probe:vfs_getname with this format is in place: # perf probe -l probe:vfs_getname (on getname_flags:72@/home/git/linux/fs/namei.c with pathname) That was, in this case, put in place with: # perf probe 'vfs_getname=getname_flags:72 pathname=filename:string' Added new event: probe:vfs_getname (on getname_flags:72 with pathname=filename:string) You can now use it in all perf tools, such as: perf record -e probe:vfs_getname -aR sleep 1 # Then 'perf trace' will notice that and do the pointer -> contents expansion: # trace -e open touch /tmp/bla 0.165 (0.010 ms): touch/17752 open(filename: /etc/ld.so.cache, flags: CLOEXEC) = 3 0.195 (0.011 ms): touch/17752 open(filename: /lib64/libc.so.6, flags: CLOEXEC) = 3 0.512 (0.012 ms): touch/17752 open(filename: /usr/lib/locale/locale-archive, flags: CLOEXEC) = 3 0.582 (0.012 ms): touch/17752 open(filename: /tmp/bla, flags: CREAT|NOCTTY|NONBLOCK|WRONLY, mode: 438) = 3 # Roughly equivalent to strace's output: # strace -rT -e open touch /tmp/bla 0.000000 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 <0.000039> 0.000317 open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 <0.000102> 0.001461 open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3 <0.000072> 0.000405 open("/tmp/bla", O_WRONLY|O_CREAT|O_NOCTTY|O_NONBLOCK, 0666) = 3 <0.000055> 0.000641 +++ exited with 0 +++ # Now we need to either look for at all syscalls that are marked as pointers and have some well known names ("filename", "pathname", etc) and set the arg formatter to the one used for the "open" syscall in this patch. This implementation works for syscalls with just a string being copied from userspace, for matching syscalls with more than one string being copied via the same probe/trace point (vfs_getname) we need to extend the vfs_getname probe spec to include the pointer too, but there are some problems with that in 'perf probe' or the kernel kprobes code, need to investigate before considering supporting multiple strings per syscall. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Milian Wolff <mail@milianw.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-xvuwx6nuj8cf389kf9s2ue2s@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-08-04 22:30:09 -03:00
trace: Beautify perf_event_open syscall Syswide tracing and then running 'stat' and 'trace': $ perf trace -e perf_event_open 1034.649 (0.019 ms): perf/6133 perf_event_open(attr_uptr: 0x36f0360, pid: 16134, cpu: -1, group_fd: -1, flags: FD_CLOEXEC) = -1 EINVAL Invalid argument 1034.670 (0.008 ms): perf/6133 perf_event_open(attr_uptr: 0x36f0360, pid: 16134, cpu: -1, group_fd: -1) = -1 EINVAL Invalid argument 1034.681 (0.007 ms): perf/6133 perf_event_open(attr_uptr: 0x36f0360, pid: 16134, cpu: -1, group_fd: -1) = -1 EINVAL Invalid argument 1034.692 (0.007 ms): perf/6133 perf_event_open(attr_uptr: 0x36f0360, pid: 16134, cpu: -1, group_fd: -1) = -1 EINVAL Invalid argument 9986.983 (0.014 ms): trace/6139 perf_event_open(attr_uptr: 0x7ffd9c629320, pid: -1, group_fd: -1, flags: FD_CLOEXEC) = 3 9987.026 (0.016 ms): trace/6139 perf_event_open(attr_uptr: 0x37c7e70, pid: -1, group_fd: -1, flags: FD_CLOEXEC) = 3 9987.041 (0.008 ms): trace/6139 perf_event_open(attr_uptr: 0x37c7e70, pid: -1, group_fd: -1, flags: FD_CLOEXEC) = 3 9987.489 (0.092 ms): trace/6139 perf_event_open(attr_uptr: 0x3795ee0, pid: 16140, group_fd: -1, flags: FD_CLOEXEC) = 3 9987.536 (0.044 ms): trace/6139 perf_event_open(attr_uptr: 0x3795ee0, pid: 16140, cpu: 1, group_fd: -1, flags: FD_CLOEXEC) = 4 9987.580 (0.041 ms): trace/6139 perf_event_open(attr_uptr: 0x3795ee0, pid: 16140, cpu: 2, group_fd: -1, flags: FD_CLOEXEC) = 5 9987.620 (0.037 ms): trace/6139 perf_event_open(attr_uptr: 0x3795ee0, pid: 16140, cpu: 3, group_fd: -1, flags: FD_CLOEXEC) = 7 9987.659 (0.035 ms): trace/6139 perf_event_open(attr_uptr: 0x37975d0, pid: 16140, group_fd: -1, flags: FD_CLOEXEC) = 8 9987.692 (0.031 ms): trace/6139 perf_event_open(attr_uptr: 0x37975d0, pid: 16140, cpu: 1, group_fd: -1, flags: FD_CLOEXEC) = 9 9987.727 (0.032 ms): trace/6139 perf_event_open(attr_uptr: 0x37975d0, pid: 16140, cpu: 2, group_fd: -1, flags: FD_CLOEXEC) = 10 9987.761 (0.031 ms): trace/6139 perf_event_open(attr_uptr: 0x37975d0, pid: 16140, cpu: 3, group_fd: -1, flags: FD_CLOEXEC) = 11 Need to intercept perf_copy_attr() with a kprobe or with eBPF... Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Link: http://lkml.kernel.org/n/tip-njb105hab2i3t5dexym9lskl@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-06-11 22:47:54 -03:00
perf trace: Initial beautifier for ioctl's 'cmd' arg [root@zoo linux]# trace -e ioctl | grep -v "cmd: 0x" | head -10 0.386 ( 0.001 ms): trace/1602 ioctl(fd: 1<pipe:[127057]>, cmd: TCGETS, arg: 0x7fff59fcb4d0 ) = -1 ENOTTY Inappropriate ioctl for device 1459.368 ( 0.002 ms): inotify_reader/10352 ioctl(fd: 18<anon_inode:inotify>, cmd: FIONREAD, arg: 0x7fb835228bcc ) = 0 1463.586 ( 0.002 ms): inotify_reader/10352 ioctl(fd: 18<anon_inode:inotify>, cmd: FIONREAD, arg: 0x7fb835228bcc ) = 0 1463.611 ( 0.002 ms): inotify_reader/10352 ioctl(fd: 18<anon_inode:inotify>, cmd: FIONREAD, arg: 0x7fb835228bcc ) = 0 3740.526 ( 0.002 ms): awk/1612 ioctl(fd: 1<pipe:[128265]>, cmd: TCGETS, arg: 0x7fff4d166b90 ) = -1 ENOTTY Inappropriate ioctl for device 3740.704 ( 0.001 ms): awk/1612 ioctl(fd: 3</proc/meminfo>, cmd: TCGETS, arg: 0x7fff4d1669a0 ) = -1 ENOTTY Inappropriate ioctl for device 3742.550 ( 0.002 ms): ps/1614 ioctl(fd: 1<pipe:[128266]>, cmd: TIOCGWINSZ, arg: 0x7fff591762b0 ) = -1 ENOTTY Inappropriate ioctl for device 3742.555 ( 0.003 ms): ps/1614 ioctl(fd: 2<socket:[19550]>, cmd: TIOCGWINSZ, arg: 0x7fff591762b0 ) = -1 ENOTTY Inappropriate ioctl for device 3742.558 ( 0.002 ms): ps/1614 ioctl(cmd: TIOCGWINSZ, arg: 0x7fff591762b0 ) = -1 ENOTTY Inappropriate ioctl for device 3742.572 ( 0.002 ms): ps/1614 ioctl(fd: 1<pipe:[128266]>, cmd: TCGETS, arg: 0x7fff59176220 ) = -1 ENOTTY Inappropriate ioctl for device [root@zoo linux]# Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-afajwap3mr60dfl4qpdl1pxn@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-08 17:43:20 -03:00
perf trace: Initial beautifier for ioctl's 'cmd' arg [root@zoo linux]# trace -e ioctl | grep -v "cmd: 0x" | head -10 0.386 ( 0.001 ms): trace/1602 ioctl(fd: 1<pipe:[127057]>, cmd: TCGETS, arg: 0x7fff59fcb4d0 ) = -1 ENOTTY Inappropriate ioctl for device 1459.368 ( 0.002 ms): inotify_reader/10352 ioctl(fd: 18<anon_inode:inotify>, cmd: FIONREAD, arg: 0x7fb835228bcc ) = 0 1463.586 ( 0.002 ms): inotify_reader/10352 ioctl(fd: 18<anon_inode:inotify>, cmd: FIONREAD, arg: 0x7fb835228bcc ) = 0 1463.611 ( 0.002 ms): inotify_reader/10352 ioctl(fd: 18<anon_inode:inotify>, cmd: FIONREAD, arg: 0x7fb835228bcc ) = 0 3740.526 ( 0.002 ms): awk/1612 ioctl(fd: 1<pipe:[128265]>, cmd: TCGETS, arg: 0x7fff4d166b90 ) = -1 ENOTTY Inappropriate ioctl for device 3740.704 ( 0.001 ms): awk/1612 ioctl(fd: 3</proc/meminfo>, cmd: TCGETS, arg: 0x7fff4d1669a0 ) = -1 ENOTTY Inappropriate ioctl for device 3742.550 ( 0.002 ms): ps/1614 ioctl(fd: 1<pipe:[128266]>, cmd: TIOCGWINSZ, arg: 0x7fff591762b0 ) = -1 ENOTTY Inappropriate ioctl for device 3742.555 ( 0.003 ms): ps/1614 ioctl(fd: 2<socket:[19550]>, cmd: TIOCGWINSZ, arg: 0x7fff591762b0 ) = -1 ENOTTY Inappropriate ioctl for device 3742.558 ( 0.002 ms): ps/1614 ioctl(cmd: TIOCGWINSZ, arg: 0x7fff591762b0 ) = -1 ENOTTY Inappropriate ioctl for device 3742.572 ( 0.002 ms): ps/1614 ioctl(fd: 1<pipe:[128266]>, cmd: TCGETS, arg: 0x7fff59176220 ) = -1 ENOTTY Inappropriate ioctl for device [root@zoo linux]# Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-afajwap3mr60dfl4qpdl1pxn@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-08 17:43:20 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Initial beautifier for ioctl's 'cmd' arg [root@zoo linux]# trace -e ioctl | grep -v "cmd: 0x" | head -10 0.386 ( 0.001 ms): trace/1602 ioctl(fd: 1<pipe:[127057]>, cmd: TCGETS, arg: 0x7fff59fcb4d0 ) = -1 ENOTTY Inappropriate ioctl for device 1459.368 ( 0.002 ms): inotify_reader/10352 ioctl(fd: 18<anon_inode:inotify>, cmd: FIONREAD, arg: 0x7fb835228bcc ) = 0 1463.586 ( 0.002 ms): inotify_reader/10352 ioctl(fd: 18<anon_inode:inotify>, cmd: FIONREAD, arg: 0x7fb835228bcc ) = 0 1463.611 ( 0.002 ms): inotify_reader/10352 ioctl(fd: 18<anon_inode:inotify>, cmd: FIONREAD, arg: 0x7fb835228bcc ) = 0 3740.526 ( 0.002 ms): awk/1612 ioctl(fd: 1<pipe:[128265]>, cmd: TCGETS, arg: 0x7fff4d166b90 ) = -1 ENOTTY Inappropriate ioctl for device 3740.704 ( 0.001 ms): awk/1612 ioctl(fd: 3</proc/meminfo>, cmd: TCGETS, arg: 0x7fff4d1669a0 ) = -1 ENOTTY Inappropriate ioctl for device 3742.550 ( 0.002 ms): ps/1614 ioctl(fd: 1<pipe:[128266]>, cmd: TIOCGWINSZ, arg: 0x7fff591762b0 ) = -1 ENOTTY Inappropriate ioctl for device 3742.555 ( 0.003 ms): ps/1614 ioctl(fd: 2<socket:[19550]>, cmd: TIOCGWINSZ, arg: 0x7fff591762b0 ) = -1 ENOTTY Inappropriate ioctl for device 3742.558 ( 0.002 ms): ps/1614 ioctl(cmd: TIOCGWINSZ, arg: 0x7fff591762b0 ) = -1 ENOTTY Inappropriate ioctl for device 3742.572 ( 0.002 ms): ps/1614 ioctl(fd: 1<pipe:[128266]>, cmd: TCGETS, arg: 0x7fff59176220 ) = -1 ENOTTY Inappropriate ioctl for device [root@zoo linux]# Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-afajwap3mr60dfl4qpdl1pxn@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-08 17:43:20 -03:00
perf trace: Beautify keyctl's option arg 8.697 (0.103 ms): pool/2343 keyctl(option: GET_PERSISTENT, arg2: 1000, arg3: 4294967294, arg4: 140703061514067, arg5: 140703692383680) = 1023192809 8.763 (0.049 ms): pool/2343 keyctl(option: SEARCH, arg2: 1023192809, arg3: 140703745767772, arg4: 140703745767832, arg5: 4294967294) = 140224497 8.789 (0.016 ms): pool/2343 keyctl(option: SEARCH, arg2: 140224497, arg3: 140703745767814, arg4: 140703745767900) = 512300257 8.807 (0.011 ms): pool/2343 keyctl(option: READ, arg2: 512300257 ) = 13 8.822 (0.008 ms): pool/2343 keyctl(option: READ, arg2: 512300257, arg3: 140703061514000, arg4: 13 ) = 13 8.837 (0.007 ms): pool/2343 keyctl(option: READ, arg2: 140224497 ) = 4 8.852 (0.009 ms): pool/2343 keyctl(option: READ, arg2: 140224497, arg3: 140703061514000, arg4: 4 ) = 4 8.869 (0.010 ms): pool/2343 keyctl(option: SEARCH, arg2: 140224497, arg3: 140703745767772, arg4: 140703061514032) = -1 ENOKEY Required key not available 8.892 (0.017 ms): pool/2343 keyctl(option: DESCRIBE, arg2: 512300257 ) = 43 8.910 (0.012 ms): pool/2343 keyctl(option: DESCRIBE, arg2: 512300257, arg3: 140703061544384, arg4: 43) = 43 Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-013ab219irsxngyumrf5gp8s@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-08-11 11:05:36 -03:00
perf trace: Add beautifier for mmap prot parm [root@zoo ~]# perf trace -e mmap,mprotect sleep 1 0.984 ( 0.015 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: 34, fd: 4294967295, off: 0) = 0xd62ae000 1.114 ( 0.016 ms): mmap(addr: 0, len: 125100, prot: READ, flags: 2, fd: 3, off: 0 ) = 0xd628f000 1.252 ( 0.020 ms): mmap(addr: 0x33c1600000, len: 3896312, prot: EXEC|READ, flags: 2050, fd: 3, off: 0) = 0xc1600000 1.282 ( 0.024 ms): mprotect(start: 0x33c17ad000, len: 2097152, prot: NONE ) = 0 1.315 ( 0.026 ms): mmap(addr: 0x33c19ad000, len: 24576, prot: READ|WRITE, flags: 2066, fd: 3, off: 1757184) = 0xc19ad000 1.352 ( 0.017 ms): mmap(addr: 0x33c19b3000, len: 17400, prot: READ|WRITE, flags: 50, fd: 4294967295, off: 0) = 0xc19b3000 1.415 ( 0.011 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: 34, fd: 4294967295, off: 0) = 0xd628e000 1.440 ( 0.011 ms): mmap(addr: 0, len: 8192, prot: READ|WRITE, flags: 34, fd: 4294967295, off: 0) = 0xd628c000 1.569 ( 0.019 ms): mprotect(start: 0x606000, len: 4096, prot: READ ) = 0 1.591 ( 0.017 ms): mprotect(start: 0x33c19ad000, len: 16384, prot: READ ) = 0 1.616 ( 0.016 ms): mprotect(start: 0x33c1420000, len: 4096, prot: READ ) = 0 2.105 ( 0.018 ms): mmap(addr: 0, len: 104789808, prot: READ, flags: 2, fd: 3, off: 0 ) = 0xcfe9c000 [root@zoo ~]# Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-q1ubhdd9wigxneam616ggdsn@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-08-20 17:44:42 -03:00
perf trace: Add beautifier for mmap flags parm [root@zoo ~]# perf trace -e mmap,mprotect sleep 1 0.992 ( 0.015 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: 4294967295, off: 0) = 0xa60be000 1.108 ( 0.012 ms): mmap(addr: 0, len: 125100, prot: READ, flags: PRIVATE, fd: 3, off: 0 ) = 0xa609f000 1.209 ( 0.014 ms): mmap(addr: 0x33c1600000, len: 3896312, prot: EXEC|READ, flags: PRIVATE|DENYWRITE, fd: 3, off: 0) = 0xc1600000 1.232 ( 0.018 ms): mprotect(start: 0x33c17ad000, len: 2097152, prot: NONE ) = 0 1.255 ( 0.018 ms): mmap(addr: 0x33c19ad000, len: 24576, prot: READ|WRITE, flags: PRIVATE|DENYWRITE|FIXED, fd: 3, off: 1757184) = 0xc19ad000 1.281 ( 0.011 ms): mmap(addr: 0x33c19b3000, len: 17400, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS|FIXED, fd: 4294967295, off: 0) = 0xc19b3000 1.328 ( 0.008 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: 4294967295, off: 0) = 0xa609e000 1.346 ( 0.008 ms): mmap(addr: 0, len: 8192, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: 4294967295, off: 0) = 0xa609c000 1.443 ( 0.013 ms): mprotect(start: 0x606000, len: 4096, prot: READ ) = 0 1.459 ( 0.011 ms): mprotect(start: 0x33c19ad000, len: 16384, prot: READ ) = 0 1.477 ( 0.011 ms): mprotect(start: 0x33c1420000, len: 4096, prot: READ ) = 0 1.855 ( 0.013 ms): mmap(addr: 0, len: 104789808, prot: READ, flags: PRIVATE, fd: 3, off: 0) = 0x9fcac000 [root@zoo ~]# Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-q1ubhdd9wigxneam616ggdsn@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-08-23 10:48:33 -03:00
perf trace: Add beautifier for mmap prot parm [root@zoo ~]# perf trace -e mmap,mprotect sleep 1 0.984 ( 0.015 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: 34, fd: 4294967295, off: 0) = 0xd62ae000 1.114 ( 0.016 ms): mmap(addr: 0, len: 125100, prot: READ, flags: 2, fd: 3, off: 0 ) = 0xd628f000 1.252 ( 0.020 ms): mmap(addr: 0x33c1600000, len: 3896312, prot: EXEC|READ, flags: 2050, fd: 3, off: 0) = 0xc1600000 1.282 ( 0.024 ms): mprotect(start: 0x33c17ad000, len: 2097152, prot: NONE ) = 0 1.315 ( 0.026 ms): mmap(addr: 0x33c19ad000, len: 24576, prot: READ|WRITE, flags: 2066, fd: 3, off: 1757184) = 0xc19ad000 1.352 ( 0.017 ms): mmap(addr: 0x33c19b3000, len: 17400, prot: READ|WRITE, flags: 50, fd: 4294967295, off: 0) = 0xc19b3000 1.415 ( 0.011 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: 34, fd: 4294967295, off: 0) = 0xd628e000 1.440 ( 0.011 ms): mmap(addr: 0, len: 8192, prot: READ|WRITE, flags: 34, fd: 4294967295, off: 0) = 0xd628c000 1.569 ( 0.019 ms): mprotect(start: 0x606000, len: 4096, prot: READ ) = 0 1.591 ( 0.017 ms): mprotect(start: 0x33c19ad000, len: 16384, prot: READ ) = 0 1.616 ( 0.016 ms): mprotect(start: 0x33c1420000, len: 4096, prot: READ ) = 0 2.105 ( 0.018 ms): mmap(addr: 0, len: 104789808, prot: READ, flags: 2, fd: 3, off: 0 ) = 0xcfe9c000 [root@zoo ~]# Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-q1ubhdd9wigxneam616ggdsn@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-08-20 17:44:42 -03:00
perf trace: Add beautifier for mmap prot parm [root@zoo ~]# perf trace -e mmap,mprotect sleep 1 0.984 ( 0.015 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: 34, fd: 4294967295, off: 0) = 0xd62ae000 1.114 ( 0.016 ms): mmap(addr: 0, len: 125100, prot: READ, flags: 2, fd: 3, off: 0 ) = 0xd628f000 1.252 ( 0.020 ms): mmap(addr: 0x33c1600000, len: 3896312, prot: EXEC|READ, flags: 2050, fd: 3, off: 0) = 0xc1600000 1.282 ( 0.024 ms): mprotect(start: 0x33c17ad000, len: 2097152, prot: NONE ) = 0 1.315 ( 0.026 ms): mmap(addr: 0x33c19ad000, len: 24576, prot: READ|WRITE, flags: 2066, fd: 3, off: 1757184) = 0xc19ad000 1.352 ( 0.017 ms): mmap(addr: 0x33c19b3000, len: 17400, prot: READ|WRITE, flags: 50, fd: 4294967295, off: 0) = 0xc19b3000 1.415 ( 0.011 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: 34, fd: 4294967295, off: 0) = 0xd628e000 1.440 ( 0.011 ms): mmap(addr: 0, len: 8192, prot: READ|WRITE, flags: 34, fd: 4294967295, off: 0) = 0xd628c000 1.569 ( 0.019 ms): mprotect(start: 0x606000, len: 4096, prot: READ ) = 0 1.591 ( 0.017 ms): mprotect(start: 0x33c19ad000, len: 16384, prot: READ ) = 0 1.616 ( 0.016 ms): mprotect(start: 0x33c1420000, len: 4096, prot: READ ) = 0 2.105 ( 0.018 ms): mmap(addr: 0, len: 104789808, prot: READ, flags: 2, fd: 3, off: 0 ) = 0xcfe9c000 [root@zoo ~]# Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-q1ubhdd9wigxneam616ggdsn@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-08-20 17:44:42 -03:00
perf trace: Add beautifier for mmap prot parm [root@zoo ~]# perf trace -e mmap,mprotect sleep 1 0.984 ( 0.015 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: 34, fd: 4294967295, off: 0) = 0xd62ae000 1.114 ( 0.016 ms): mmap(addr: 0, len: 125100, prot: READ, flags: 2, fd: 3, off: 0 ) = 0xd628f000 1.252 ( 0.020 ms): mmap(addr: 0x33c1600000, len: 3896312, prot: EXEC|READ, flags: 2050, fd: 3, off: 0) = 0xc1600000 1.282 ( 0.024 ms): mprotect(start: 0x33c17ad000, len: 2097152, prot: NONE ) = 0 1.315 ( 0.026 ms): mmap(addr: 0x33c19ad000, len: 24576, prot: READ|WRITE, flags: 2066, fd: 3, off: 1757184) = 0xc19ad000 1.352 ( 0.017 ms): mmap(addr: 0x33c19b3000, len: 17400, prot: READ|WRITE, flags: 50, fd: 4294967295, off: 0) = 0xc19b3000 1.415 ( 0.011 ms): mmap(addr: 0, len: 4096, prot: READ|WRITE, flags: 34, fd: 4294967295, off: 0) = 0xd628e000 1.440 ( 0.011 ms): mmap(addr: 0, len: 8192, prot: READ|WRITE, flags: 34, fd: 4294967295, off: 0) = 0xd628c000 1.569 ( 0.019 ms): mprotect(start: 0x606000, len: 4096, prot: READ ) = 0 1.591 ( 0.017 ms): mprotect(start: 0x33c19ad000, len: 16384, prot: READ ) = 0 1.616 ( 0.016 ms): mprotect(start: 0x33c1420000, len: 4096, prot: READ ) = 0 2.105 ( 0.018 ms): mmap(addr: 0, len: 104789808, prot: READ, flags: 2, fd: 3, off: 0 ) = 0xcfe9c000 [root@zoo ~]# Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-q1ubhdd9wigxneam616ggdsn@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-08-20 17:44:42 -03:00
perf trace: Deref sys_enter pointer args with contents from probe:vfs_getname To work like strace and dereference syscall pointer args we need to insert probes (or tracepoints) right after we copy those bytes from userspace. Since we're formatting the syscall args at raw_syscalls:sys_enter time, we need to have a formatter that just stores the position where, later, when we get the probe:vfs_getname, we can insert the pointer contents. Now, if a probe:vfs_getname with this format is in place: # perf probe -l probe:vfs_getname (on getname_flags:72@/home/git/linux/fs/namei.c with pathname) That was, in this case, put in place with: # perf probe 'vfs_getname=getname_flags:72 pathname=filename:string' Added new event: probe:vfs_getname (on getname_flags:72 with pathname=filename:string) You can now use it in all perf tools, such as: perf record -e probe:vfs_getname -aR sleep 1 # Then 'perf trace' will notice that and do the pointer -> contents expansion: # trace -e open touch /tmp/bla 0.165 (0.010 ms): touch/17752 open(filename: /etc/ld.so.cache, flags: CLOEXEC) = 3 0.195 (0.011 ms): touch/17752 open(filename: /lib64/libc.so.6, flags: CLOEXEC) = 3 0.512 (0.012 ms): touch/17752 open(filename: /usr/lib/locale/locale-archive, flags: CLOEXEC) = 3 0.582 (0.012 ms): touch/17752 open(filename: /tmp/bla, flags: CREAT|NOCTTY|NONBLOCK|WRONLY, mode: 438) = 3 # Roughly equivalent to strace's output: # strace -rT -e open touch /tmp/bla 0.000000 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 <0.000039> 0.000317 open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 <0.000102> 0.001461 open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3 <0.000072> 0.000405 open("/tmp/bla", O_WRONLY|O_CREAT|O_NOCTTY|O_NONBLOCK, 0666) = 3 <0.000055> 0.000641 +++ exited with 0 +++ # Now we need to either look for at all syscalls that are marked as pointers and have some well known names ("filename", "pathname", etc) and set the arg formatter to the one used for the "open" syscall in this patch. This implementation works for syscalls with just a string being copied from userspace, for matching syscalls with more than one string being copied via the same probe/trace point (vfs_getname) we need to extend the vfs_getname probe spec to include the pointer too, but there are some problems with that in 'perf probe' or the kernel kprobes code, need to investigate before considering supporting multiple strings per syscall. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Milian Wolff <mail@milianw.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-xvuwx6nuj8cf389kf9s2ue2s@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-08-04 22:30:09 -03:00
trace: Beautify perf_event_open syscall Syswide tracing and then running 'stat' and 'trace': $ perf trace -e perf_event_open 1034.649 (0.019 ms): perf/6133 perf_event_open(attr_uptr: 0x36f0360, pid: 16134, cpu: -1, group_fd: -1, flags: FD_CLOEXEC) = -1 EINVAL Invalid argument 1034.670 (0.008 ms): perf/6133 perf_event_open(attr_uptr: 0x36f0360, pid: 16134, cpu: -1, group_fd: -1) = -1 EINVAL Invalid argument 1034.681 (0.007 ms): perf/6133 perf_event_open(attr_uptr: 0x36f0360, pid: 16134, cpu: -1, group_fd: -1) = -1 EINVAL Invalid argument 1034.692 (0.007 ms): perf/6133 perf_event_open(attr_uptr: 0x36f0360, pid: 16134, cpu: -1, group_fd: -1) = -1 EINVAL Invalid argument 9986.983 (0.014 ms): trace/6139 perf_event_open(attr_uptr: 0x7ffd9c629320, pid: -1, group_fd: -1, flags: FD_CLOEXEC) = 3 9987.026 (0.016 ms): trace/6139 perf_event_open(attr_uptr: 0x37c7e70, pid: -1, group_fd: -1, flags: FD_CLOEXEC) = 3 9987.041 (0.008 ms): trace/6139 perf_event_open(attr_uptr: 0x37c7e70, pid: -1, group_fd: -1, flags: FD_CLOEXEC) = 3 9987.489 (0.092 ms): trace/6139 perf_event_open(attr_uptr: 0x3795ee0, pid: 16140, group_fd: -1, flags: FD_CLOEXEC) = 3 9987.536 (0.044 ms): trace/6139 perf_event_open(attr_uptr: 0x3795ee0, pid: 16140, cpu: 1, group_fd: -1, flags: FD_CLOEXEC) = 4 9987.580 (0.041 ms): trace/6139 perf_event_open(attr_uptr: 0x3795ee0, pid: 16140, cpu: 2, group_fd: -1, flags: FD_CLOEXEC) = 5 9987.620 (0.037 ms): trace/6139 perf_event_open(attr_uptr: 0x3795ee0, pid: 16140, cpu: 3, group_fd: -1, flags: FD_CLOEXEC) = 7 9987.659 (0.035 ms): trace/6139 perf_event_open(attr_uptr: 0x37975d0, pid: 16140, group_fd: -1, flags: FD_CLOEXEC) = 8 9987.692 (0.031 ms): trace/6139 perf_event_open(attr_uptr: 0x37975d0, pid: 16140, cpu: 1, group_fd: -1, flags: FD_CLOEXEC) = 9 9987.727 (0.032 ms): trace/6139 perf_event_open(attr_uptr: 0x37975d0, pid: 16140, cpu: 2, group_fd: -1, flags: FD_CLOEXEC) = 10 9987.761 (0.031 ms): trace/6139 perf_event_open(attr_uptr: 0x37975d0, pid: 16140, cpu: 3, group_fd: -1, flags: FD_CLOEXEC) = 11 Need to intercept perf_copy_attr() with a kprobe or with eBPF... Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Link: http://lkml.kernel.org/n/tip-njb105hab2i3t5dexym9lskl@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-06-11 22:47:54 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Deref sys_enter pointer args with contents from probe:vfs_getname To work like strace and dereference syscall pointer args we need to insert probes (or tracepoints) right after we copy those bytes from userspace. Since we're formatting the syscall args at raw_syscalls:sys_enter time, we need to have a formatter that just stores the position where, later, when we get the probe:vfs_getname, we can insert the pointer contents. Now, if a probe:vfs_getname with this format is in place: # perf probe -l probe:vfs_getname (on getname_flags:72@/home/git/linux/fs/namei.c with pathname) That was, in this case, put in place with: # perf probe 'vfs_getname=getname_flags:72 pathname=filename:string' Added new event: probe:vfs_getname (on getname_flags:72 with pathname=filename:string) You can now use it in all perf tools, such as: perf record -e probe:vfs_getname -aR sleep 1 # Then 'perf trace' will notice that and do the pointer -> contents expansion: # trace -e open touch /tmp/bla 0.165 (0.010 ms): touch/17752 open(filename: /etc/ld.so.cache, flags: CLOEXEC) = 3 0.195 (0.011 ms): touch/17752 open(filename: /lib64/libc.so.6, flags: CLOEXEC) = 3 0.512 (0.012 ms): touch/17752 open(filename: /usr/lib/locale/locale-archive, flags: CLOEXEC) = 3 0.582 (0.012 ms): touch/17752 open(filename: /tmp/bla, flags: CREAT|NOCTTY|NONBLOCK|WRONLY, mode: 438) = 3 # Roughly equivalent to strace's output: # strace -rT -e open touch /tmp/bla 0.000000 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 <0.000039> 0.000317 open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 <0.000102> 0.001461 open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3 <0.000072> 0.000405 open("/tmp/bla", O_WRONLY|O_CREAT|O_NOCTTY|O_NONBLOCK, 0666) = 3 <0.000055> 0.000641 +++ exited with 0 +++ # Now we need to either look for at all syscalls that are marked as pointers and have some well known names ("filename", "pathname", etc) and set the arg formatter to the one used for the "open" syscall in this patch. This implementation works for syscalls with just a string being copied from userspace, for matching syscalls with more than one string being copied via the same probe/trace point (vfs_getname) we need to extend the vfs_getname probe spec to include the pointer too, but there are some problems with that in 'perf probe' or the kernel kprobes code, need to investigate before considering supporting multiple strings per syscall. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Milian Wolff <mail@milianw.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-xvuwx6nuj8cf389kf9s2ue2s@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-08-04 22:30:09 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Use sched:sched_stat_runtime to provide a thread summary [root@sandy ~]# perf trace --sched --duration 0.100 --pid `pidof firefox` <SNIP> 17079.847 ( 0.009 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.892 ( 0.010 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable 17079.921 ( 0.013 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.949 ( 0.009 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ firefox - 17643 : 18013 [ 72.2% ] 359.110 ms firefox - 17663 : 41 [ 0.2% ] 21.439 ms firefox - 17664 : 6840 [ 27.4% ] 133.642 ms firefox - 17667 : 46 [ 0.2% ] 0.682 ms [root@sandy ~]# This is equivalent to the 'perf trace summary' subcomand in the tmp.perf/trace2 branch. Another example, setting a huge duration filter to get just a system wide summary: [root@sandy ~]# perf trace --duration 10000.0 --sched ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ scsi_eh_1 - 258 : 15 [ 0.0% ] 0.133 ms kworker/0:1H - 322 : 13 [ 0.0% ] 0.032 ms jbd2/dm-0-8 - 384 : 4 [ 0.0% ] 0.115 ms flush-253:0 - 470 : 1 [ 0.0% ] 0.027 ms firefox - 950 : 4783 [ 0.1% ] 24.863 ms firefox - 992 : 1883 [ 0.1% ] 6.808 ms firefox - 995 : 35 [ 0.0% ] 0.111 ms ksoftirqd/6 - 4362 : 2 [ 0.0% ] 0.005 ms ksoftirqd/7 - 4365 : 1 [ 0.0% ] 0.007 ms Xorg - 4671 : 148 [ 0.0% ] 0.912 ms gnome-settings- - 4846 : 14 [ 0.0% ] 0.086 ms seahorse-daemon - 4847 : 14 [ 0.0% ] 0.092 ms gnome-panel - 4875 : 46 [ 0.0% ] 0.159 ms gnome-power-man - 4918 : 16 [ 0.0% ] 0.065 ms gvfs-afc-volume - 4992 : 77 [ 0.0% ] 0.136 ms gnome-screensav - 5114 : 24 [ 0.0% ] 0.128 ms xchat - 8082 : 466 [ 0.0% ] 2.019 ms synergyc - 8369 : 941 [ 0.0% ] 3.291 ms synergyc - 8371 : 85 [ 0.0% ] 1.817 ms jbd2/dm-4-8 - 9352 : 4 [ 0.0% ] 0.109 ms rpcbind - 9786 : 3 [ 0.0% ] 0.017 ms rtkit-daemon - 12802 : 10 [ 0.0% ] 0.038 ms rtkit-daemon - 12803 : 8 [ 0.0% ] 0.000 ms udisks-daemon - 13020 : 27 [ 0.0% ] 0.240 ms kworker/7:0 - 14651 : 669 [ 0.0% ] 2.616 ms kworker/5:1 - 16220 : 2 [ 0.0% ] 0.069 ms kworker/4:0 - 19776 : 13 [ 0.0% ] 0.176 ms openvpn - 20131 : 133 [ 0.0% ] 0.762 ms plugin-containe - 20508 : 60658 [ 1.7% ] 131.153 ms npviewer.bin - 20520 : 72208 [ 2.0% ] 138.945 ms npviewer.bin - 20542 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 20543 : 30 [ 0.0% ] 0.074 ms npviewer.bin - 20547 : 35 [ 0.0% ] 0.092 ms npviewer.bin - 20552 : 35 [ 0.0% ] 0.093 ms sshd - 20645 : 32 [ 0.0% ] 0.071 ms npviewer.bin - 21053 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 21054 : 35 [ 0.0% ] 0.097 ms kworker/0:2 - 21169 : 149 [ 0.0% ] 1.143 ms kworker/3:0 - 22171 : 113 [ 0.0% ] 96.892 ms flush-253:4 - 22410 : 1 [ 0.0% ] 0.028 ms kworker/6:0 - 24581 : 25 [ 0.0% ] 0.275 ms kworker/1:0 - 25572 : 4 [ 0.0% ] 0.103 ms kworker/2:1 - 26299 : 138 [ 0.0% ] 1.440 ms kworker/0:0 - 26325 : 1 [ 0.0% ] 0.003 ms perf - 26330 : 3506967 [ 96.1% ] 6648.310 ms [root@sandy ~]# Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-mzuli0srnxyi1o029py6537x@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-17 17:13:12 -03:00
perf trace: Deref sys_enter pointer args with contents from probe:vfs_getname To work like strace and dereference syscall pointer args we need to insert probes (or tracepoints) right after we copy those bytes from userspace. Since we're formatting the syscall args at raw_syscalls:sys_enter time, we need to have a formatter that just stores the position where, later, when we get the probe:vfs_getname, we can insert the pointer contents. Now, if a probe:vfs_getname with this format is in place: # perf probe -l probe:vfs_getname (on getname_flags:72@/home/git/linux/fs/namei.c with pathname) That was, in this case, put in place with: # perf probe 'vfs_getname=getname_flags:72 pathname=filename:string' Added new event: probe:vfs_getname (on getname_flags:72 with pathname=filename:string) You can now use it in all perf tools, such as: perf record -e probe:vfs_getname -aR sleep 1 # Then 'perf trace' will notice that and do the pointer -> contents expansion: # trace -e open touch /tmp/bla 0.165 (0.010 ms): touch/17752 open(filename: /etc/ld.so.cache, flags: CLOEXEC) = 3 0.195 (0.011 ms): touch/17752 open(filename: /lib64/libc.so.6, flags: CLOEXEC) = 3 0.512 (0.012 ms): touch/17752 open(filename: /usr/lib/locale/locale-archive, flags: CLOEXEC) = 3 0.582 (0.012 ms): touch/17752 open(filename: /tmp/bla, flags: CREAT|NOCTTY|NONBLOCK|WRONLY, mode: 438) = 3 # Roughly equivalent to strace's output: # strace -rT -e open touch /tmp/bla 0.000000 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 <0.000039> 0.000317 open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 <0.000102> 0.001461 open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3 <0.000072> 0.000405 open("/tmp/bla", O_WRONLY|O_CREAT|O_NOCTTY|O_NONBLOCK, 0666) = 3 <0.000055> 0.000641 +++ exited with 0 +++ # Now we need to either look for at all syscalls that are marked as pointers and have some well known names ("filename", "pathname", etc) and set the arg formatter to the one used for the "open" syscall in this patch. This implementation works for syscalls with just a string being copied from userspace, for matching syscalls with more than one string being copied via the same probe/trace point (vfs_getname) we need to extend the vfs_getname probe spec to include the pointer too, but there are some problems with that in 'perf probe' or the kernel kprobes code, need to investigate before considering supporting multiple strings per syscall. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Milian Wolff <mail@milianw.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-xvuwx6nuj8cf389kf9s2ue2s@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-08-04 22:30:09 -03:00
perf trace: Deref sys_enter pointer args with contents from probe:vfs_getname To work like strace and dereference syscall pointer args we need to insert probes (or tracepoints) right after we copy those bytes from userspace. Since we're formatting the syscall args at raw_syscalls:sys_enter time, we need to have a formatter that just stores the position where, later, when we get the probe:vfs_getname, we can insert the pointer contents. Now, if a probe:vfs_getname with this format is in place: # perf probe -l probe:vfs_getname (on getname_flags:72@/home/git/linux/fs/namei.c with pathname) That was, in this case, put in place with: # perf probe 'vfs_getname=getname_flags:72 pathname=filename:string' Added new event: probe:vfs_getname (on getname_flags:72 with pathname=filename:string) You can now use it in all perf tools, such as: perf record -e probe:vfs_getname -aR sleep 1 # Then 'perf trace' will notice that and do the pointer -> contents expansion: # trace -e open touch /tmp/bla 0.165 (0.010 ms): touch/17752 open(filename: /etc/ld.so.cache, flags: CLOEXEC) = 3 0.195 (0.011 ms): touch/17752 open(filename: /lib64/libc.so.6, flags: CLOEXEC) = 3 0.512 (0.012 ms): touch/17752 open(filename: /usr/lib/locale/locale-archive, flags: CLOEXEC) = 3 0.582 (0.012 ms): touch/17752 open(filename: /tmp/bla, flags: CREAT|NOCTTY|NONBLOCK|WRONLY, mode: 438) = 3 # Roughly equivalent to strace's output: # strace -rT -e open touch /tmp/bla 0.000000 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 <0.000039> 0.000317 open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 <0.000102> 0.001461 open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3 <0.000072> 0.000405 open("/tmp/bla", O_WRONLY|O_CREAT|O_NOCTTY|O_NONBLOCK, 0666) = 3 <0.000055> 0.000641 +++ exited with 0 +++ # Now we need to either look for at all syscalls that are marked as pointers and have some well known names ("filename", "pathname", etc) and set the arg formatter to the one used for the "open" syscall in this patch. This implementation works for syscalls with just a string being copied from userspace, for matching syscalls with more than one string being copied via the same probe/trace point (vfs_getname) we need to extend the vfs_getname probe spec to include the pointer too, but there are some problems with that in 'perf probe' or the kernel kprobes code, need to investigate before considering supporting multiple strings per syscall. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Milian Wolff <mail@milianw.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-xvuwx6nuj8cf389kf9s2ue2s@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-08-04 22:30:09 -03:00
perf trace: Add summary option to dump syscall statistics When enabled dumps a summary of all syscalls by task with the usual statistics -- min, max, average and relative stddev. For example, make - 26341 : 3344 [ 17.4% ] 0.000 ms read : 52 0.000 4.802 0.644 30.08 write : 20 0.004 0.036 0.010 21.72 open : 24 0.003 0.046 0.014 23.68 close : 64 0.002 0.055 0.008 22.53 stat : 2714 0.002 0.222 0.004 4.47 fstat : 18 0.001 0.041 0.006 46.26 mmap : 30 0.003 0.009 0.006 5.71 mprotect : 8 0.006 0.039 0.016 32.16 munmap : 12 0.007 0.077 0.020 38.25 brk : 48 0.002 0.014 0.004 10.18 rt_sigaction : 18 0.002 0.002 0.002 2.11 rt_sigprocmask : 60 0.002 0.128 0.010 32.88 access : 2 0.006 0.006 0.006 0.00 pipe : 12 0.004 0.048 0.013 35.98 vfork : 34 0.448 0.980 0.692 3.04 execve : 20 0.000 0.387 0.046 56.66 wait4 : 34 0.017 9923.287 593.221 68.45 fcntl : 8 0.001 0.041 0.013 48.79 getdents : 48 0.002 0.079 0.013 19.62 getcwd : 2 0.005 0.005 0.005 0.00 chdir : 2 0.070 0.070 0.070 0.00 getrlimit : 2 0.045 0.045 0.045 0.00 arch_prctl : 2 0.002 0.002 0.002 0.00 setrlimit : 2 0.002 0.002 0.002 0.00 openat : 94 0.003 0.005 0.003 2.11 Signed-off-by: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung.kim@lge.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1381289214-24885-3-git-send-email-dsahern@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-08 21:26:53 -06:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Add summary option to dump syscall statistics When enabled dumps a summary of all syscalls by task with the usual statistics -- min, max, average and relative stddev. For example, make - 26341 : 3344 [ 17.4% ] 0.000 ms read : 52 0.000 4.802 0.644 30.08 write : 20 0.004 0.036 0.010 21.72 open : 24 0.003 0.046 0.014 23.68 close : 64 0.002 0.055 0.008 22.53 stat : 2714 0.002 0.222 0.004 4.47 fstat : 18 0.001 0.041 0.006 46.26 mmap : 30 0.003 0.009 0.006 5.71 mprotect : 8 0.006 0.039 0.016 32.16 munmap : 12 0.007 0.077 0.020 38.25 brk : 48 0.002 0.014 0.004 10.18 rt_sigaction : 18 0.002 0.002 0.002 2.11 rt_sigprocmask : 60 0.002 0.128 0.010 32.88 access : 2 0.006 0.006 0.006 0.00 pipe : 12 0.004 0.048 0.013 35.98 vfork : 34 0.448 0.980 0.692 3.04 execve : 20 0.000 0.387 0.046 56.66 wait4 : 34 0.017 9923.287 593.221 68.45 fcntl : 8 0.001 0.041 0.013 48.79 getdents : 48 0.002 0.079 0.013 19.62 getcwd : 2 0.005 0.005 0.005 0.00 chdir : 2 0.070 0.070 0.070 0.00 getrlimit : 2 0.045 0.045 0.045 0.00 arch_prctl : 2 0.002 0.002 0.002 0.00 setrlimit : 2 0.002 0.002 0.002 0.00 openat : 94 0.003 0.005 0.003 2.11 Signed-off-by: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung.kim@lge.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1381289214-24885-3-git-send-email-dsahern@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-08 21:26:53 -06:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Allow mixing with other events Basically adopting 'perf record' --event command line argument syntax: # trace -e \!mprotect,mmap,munmap,open,close,read,fstat,access,arch_prctl --event sched:*switch,sched:*exec,sched:*exit usleep 1 0.048 ( ): sched:sched_process_exec:filename=/bin/usleep pid=24732 old_pid=24732) 0.078 (0.002 ms): usleep/24732 brk( ) = 0x78f000 0.430 (0.002 ms): usleep/24732 brk( ) = 0x78f000 0.434 (0.003 ms): usleep/24732 brk(brk: 0x7b0000 ) = 0x7b0000 0.438 (0.001 ms): usleep/24732 brk( ) = 0x7b0000 0.460 (0.004 ms): usleep/24732 nanosleep(rqtp: 0x7ffff3696a40) ... 0.460 ( ): sched:sched_switch:prev_comm=usleep prev_pid=24732 prev_prio=120 prev_state=S ==> next_comm=swapper/1 next_pid=0 next_prio=120) 0.515 (0.058 ms): usleep/24732 ... [continued]: nanosleep()) = 0 0.520 (0.000 ms): usleep/24732 exit_group( 0.550 ( ): sched:sched_process_exit:comm=usleep pid=24732 prio=120) # Next steps, probably in this order: 1) Use ordered_events code, the logic in trace needs the events to be time ordered when needed, i.e. when multiple CPUs are involved. 2) Callchains! 3) Automatically account for interruptions when saying how long things took. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-gpst8mph575yb4wgf91qibyb@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-03 12:58:57 -03:00
perf trace: Handle multiple threads better wrt syscalls being intermixed $ trace time taskset -c 0 usleep 1 0.845 ( 0.021 ms): time/16722 wait4(upid: 4294967295, stat_addr: 0x7fff17f443d4, ru: 0x7fff17f44438 ) ... 0.865 ( 0.008 ms): time/16723 execve(arg0: 140733595272004, arg1: 140733595272720, arg2: 140733595272768, arg3: 139755107218496, arg4: 7307199665339051828, arg5: 3) = -2 2.395 ( 1.523 ms): taskset/16723 execve(arg0: 140733595272013, arg1: 140733595272720, arg2: 140733595272768, arg3: 139755107218496, arg4: 7307199665339051828, arg5: 3) = 0 2.411 ( 0.002 ms): taskset/16723 brk( ) = 0x1915000 3.300 ( 0.058 ms): usleep/16723 nanosleep(rqtp: 0x7ffff4ada190 ) = 0 <SNIP> 3.305 ( 0.000 ms): usleep/16723 exit_group( 3.363 ( 2.539 ms): time/16722 ... [continued]: wait4()) = 16723 3.366 ( 0.001 ms): time/16722 rt_sigaction(sig: INT, act: 0x7fff17f44160, oact: 0x7fff17f44200, sigsetsize: 8) = 0 We we're not seeing this line: 0.845 ( 0.021 ms): time/16722 wait4(upid: 4294967295, stat_addr: 0x7fff17f443d4, ru: 0x7fff17f44438 ) ... just the one when it finishes: 3.363 ( 2.539 ms): time/16722 ... [continued]: wait4()) = 16723 Still some issues left till we move to ordered_samples when multiple CPUs/threads are involved... Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-zq9x30a1ky3djqewqn2v3ja3@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-13 13:22:21 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Introduce --filter-pids When tracing in X we get event loops due to the tracing activity, i.e. updates to a gnome-terminal that generate syscalls for X.org, etc. To get a more useful view of what is happening, syscall wise, system wide, we need to filter those, like in: # ps ax|egrep '981|2296|1519' | grep -v egrep 981 tty1 Ss+ 5:40 /usr/bin/Xorg :0 -background none ... 1519 ? Sl 2:22 /usr/bin/gnome-shell 2296 ? Sl 4:16 /usr/libexec/gnome-terminal-server # # trace -e write --filter-pids 981,2296,1519 0.385 ( 0.021 ms): goa-daemon/2061 write(fd: 1</dev/null>, buf: 0x7fbeb017b000, count: 136) = 136 0.922 ( 0.014 ms): goa-daemon/2061 write(fd: 1</dev/null>, buf: 0x7fbeb017b000, count: 140) = 140 5006.525 ( 0.029 ms): goa-daemon/2061 write(fd: 1</dev/null>, buf: 0x7fbeb017b000, count: 136) = 136 5007.235 ( 0.023 ms): goa-daemon/2061 write(fd: 1</dev/null>, buf: 0x7fbeb017b000, count: 140) = 140 5177.646 ( 0.018 ms): rtkit-daemon/782 write(fd: 5<anon_inode:[eventfd]>, buf: 0x7f7eea70be88, count: 8) = 8 8314.497 ( 0.004 ms): gsd-locate-poi/2084 write(fd: 5<anon_inode:[eventfd]>, buf: 0x7fffe96af7b0, count: 8) = 8 8314.518 ( 0.002 ms): gsd-locate-poi/2084 write(fd: 5<anon_inode:[eventfd]>, buf: 0x7fffe96af0e0, count: 8) = 8 ^C# When this option is used the tracer pid is also filtered. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-f5qmiyy7c0uxdm21ncatpeek@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-21 11:36:52 -08:00
perf trace: Use sched:sched_stat_runtime to provide a thread summary [root@sandy ~]# perf trace --sched --duration 0.100 --pid `pidof firefox` <SNIP> 17079.847 ( 0.009 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.892 ( 0.010 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable 17079.921 ( 0.013 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.949 ( 0.009 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ firefox - 17643 : 18013 [ 72.2% ] 359.110 ms firefox - 17663 : 41 [ 0.2% ] 21.439 ms firefox - 17664 : 6840 [ 27.4% ] 133.642 ms firefox - 17667 : 46 [ 0.2% ] 0.682 ms [root@sandy ~]# This is equivalent to the 'perf trace summary' subcomand in the tmp.perf/trace2 branch. Another example, setting a huge duration filter to get just a system wide summary: [root@sandy ~]# perf trace --duration 10000.0 --sched ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ scsi_eh_1 - 258 : 15 [ 0.0% ] 0.133 ms kworker/0:1H - 322 : 13 [ 0.0% ] 0.032 ms jbd2/dm-0-8 - 384 : 4 [ 0.0% ] 0.115 ms flush-253:0 - 470 : 1 [ 0.0% ] 0.027 ms firefox - 950 : 4783 [ 0.1% ] 24.863 ms firefox - 992 : 1883 [ 0.1% ] 6.808 ms firefox - 995 : 35 [ 0.0% ] 0.111 ms ksoftirqd/6 - 4362 : 2 [ 0.0% ] 0.005 ms ksoftirqd/7 - 4365 : 1 [ 0.0% ] 0.007 ms Xorg - 4671 : 148 [ 0.0% ] 0.912 ms gnome-settings- - 4846 : 14 [ 0.0% ] 0.086 ms seahorse-daemon - 4847 : 14 [ 0.0% ] 0.092 ms gnome-panel - 4875 : 46 [ 0.0% ] 0.159 ms gnome-power-man - 4918 : 16 [ 0.0% ] 0.065 ms gvfs-afc-volume - 4992 : 77 [ 0.0% ] 0.136 ms gnome-screensav - 5114 : 24 [ 0.0% ] 0.128 ms xchat - 8082 : 466 [ 0.0% ] 2.019 ms synergyc - 8369 : 941 [ 0.0% ] 3.291 ms synergyc - 8371 : 85 [ 0.0% ] 1.817 ms jbd2/dm-4-8 - 9352 : 4 [ 0.0% ] 0.109 ms rpcbind - 9786 : 3 [ 0.0% ] 0.017 ms rtkit-daemon - 12802 : 10 [ 0.0% ] 0.038 ms rtkit-daemon - 12803 : 8 [ 0.0% ] 0.000 ms udisks-daemon - 13020 : 27 [ 0.0% ] 0.240 ms kworker/7:0 - 14651 : 669 [ 0.0% ] 2.616 ms kworker/5:1 - 16220 : 2 [ 0.0% ] 0.069 ms kworker/4:0 - 19776 : 13 [ 0.0% ] 0.176 ms openvpn - 20131 : 133 [ 0.0% ] 0.762 ms plugin-containe - 20508 : 60658 [ 1.7% ] 131.153 ms npviewer.bin - 20520 : 72208 [ 2.0% ] 138.945 ms npviewer.bin - 20542 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 20543 : 30 [ 0.0% ] 0.074 ms npviewer.bin - 20547 : 35 [ 0.0% ] 0.092 ms npviewer.bin - 20552 : 35 [ 0.0% ] 0.093 ms sshd - 20645 : 32 [ 0.0% ] 0.071 ms npviewer.bin - 21053 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 21054 : 35 [ 0.0% ] 0.097 ms kworker/0:2 - 21169 : 149 [ 0.0% ] 1.143 ms kworker/3:0 - 22171 : 113 [ 0.0% ] 96.892 ms flush-253:4 - 22410 : 1 [ 0.0% ] 0.028 ms kworker/6:0 - 24581 : 25 [ 0.0% ] 0.275 ms kworker/1:0 - 25572 : 4 [ 0.0% ] 0.103 ms kworker/2:1 - 26299 : 138 [ 0.0% ] 1.440 ms kworker/0:0 - 26325 : 1 [ 0.0% ] 0.003 ms perf - 26330 : 3506967 [ 96.1% ] 6648.310 ms [root@sandy ~]# Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-mzuli0srnxyi1o029py6537x@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-17 17:13:12 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Add summary option to dump syscall statistics When enabled dumps a summary of all syscalls by task with the usual statistics -- min, max, average and relative stddev. For example, make - 26341 : 3344 [ 17.4% ] 0.000 ms read : 52 0.000 4.802 0.644 30.08 write : 20 0.004 0.036 0.010 21.72 open : 24 0.003 0.046 0.014 23.68 close : 64 0.002 0.055 0.008 22.53 stat : 2714 0.002 0.222 0.004 4.47 fstat : 18 0.001 0.041 0.006 46.26 mmap : 30 0.003 0.009 0.006 5.71 mprotect : 8 0.006 0.039 0.016 32.16 munmap : 12 0.007 0.077 0.020 38.25 brk : 48 0.002 0.014 0.004 10.18 rt_sigaction : 18 0.002 0.002 0.002 2.11 rt_sigprocmask : 60 0.002 0.128 0.010 32.88 access : 2 0.006 0.006 0.006 0.00 pipe : 12 0.004 0.048 0.013 35.98 vfork : 34 0.448 0.980 0.692 3.04 execve : 20 0.000 0.387 0.046 56.66 wait4 : 34 0.017 9923.287 593.221 68.45 fcntl : 8 0.001 0.041 0.013 48.79 getdents : 48 0.002 0.079 0.013 19.62 getcwd : 2 0.005 0.005 0.005 0.00 chdir : 2 0.070 0.070 0.070 0.00 getrlimit : 2 0.045 0.045 0.045 0.00 arch_prctl : 2 0.002 0.002 0.002 0.00 setrlimit : 2 0.002 0.002 0.002 0.00 openat : 94 0.003 0.005 0.003 2.11 Signed-off-by: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung.kim@lge.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1381289214-24885-3-git-send-email-dsahern@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-08 21:26:53 -06:00
perf trace: Add option to show process COMM Enabled by default, disable with --no-comm, e.g.: 181.821 (0.001 ms): deja-dup-monit/10784 recvmsg(fd: 8, msg: 0x7fff4342baf0, flags: PEEK|TRUNC|CMSG_CLOEXEC ) = 20 181.824 (0.001 ms): deja-dup-monit/10784 geteuid( ) = 1000 181.825 (0.001 ms): deja-dup-monit/10784 getegid( ) = 1000 181.834 (0.002 ms): deja-dup-monit/10784 recvmsg(fd: 8, msg: 0x7fff4342baf0, flags: CMSG_CLOEXEC ) = 20 181.836 (0.001 ms): deja-dup-monit/10784 geteuid( ) = 1000 181.838 (0.001 ms): deja-dup-monit/10784 getegid( ) = 1000 181.705 (0.003 ms): evolution-addr/10924 recvmsg(fd: 10, msg: 0x7fff17dc6990, flags: PEEK|TRUNC|CMSG_CLOEXEC) = 1256 181.710 (0.002 ms): evolution-addr/10924 geteuid( ) = 1000 181.712 (0.001 ms): evolution-addr/10924 getegid( ) = 1000 181.727 (0.003 ms): evolution-addr/10924 recvmsg(fd: 10, msg: 0x7fff17dc6990, flags: CMSG_CLOEXEC ) = 1256 181.731 (0.001 ms): evolution-addr/10924 geteuid( ) = 1000 181.734 (0.001 ms): evolution-addr/10924 getegid( ) = 1000 181.908 (0.002 ms): evolution-addr/10924 recvmsg(fd: 10, msg: 0x7fff17dc6990, flags: PEEK|TRUNC|CMSG_CLOEXEC) = 20 181.913 (0.001 ms): evolution-addr/10924 geteuid( ) = 1000 181.915 (0.001 ms): evolution-addr/10924 getegid( ) = 1000 181.930 (0.003 ms): evolution-addr/10924 recvmsg(fd: 10, msg: 0x7fff17dc6990, flags: CMSG_CLOEXEC ) = 20 181.934 (0.001 ms): evolution-addr/10924 geteuid( ) = 1000 181.937 (0.001 ms): evolution-addr/10924 getegid( ) = 1000 220.718 (0.010 ms): at-spi2-regist/10715 sendmsg(fd: 3, msg: 0x7fffdb8756c0, flags: NOSIGNAL ) = 200 220.741 (0.000 ms): dbus-daemon/10711 ... [continued]: epoll_wait()) = 1 220.759 (0.004 ms): dbus-daemon/10711 recvmsg(fd: 11, msg: 0x7ffff94594d0, flags: CMSG_CLOEXEC ) = 200 220.780 (0.002 ms): dbus-daemon/10711 recvmsg(fd: 11, msg: 0x7ffff94594d0, flags: CMSG_CLOEXEC ) = 200 220.788 (0.001 ms): dbus-daemon/10711 recvmsg(fd: 11, msg: 0x7ffff94594d0, flags: CMSG_CLOEXEC ) = -1 EAGAIN Resource temporarily unavailable 220.760 (0.004 ms): at-spi2-regist/10715 sendmsg(fd: 3, msg: 0x7fffdb8756c0, flags: NOSIGNAL ) = 200 220.771 (0.023 ms): perf/26347 open(filename: 0xf2e780, mode: 15918976 ) = 19 220.850 (0.002 ms): perf/26347 close(fd: 19 ) = 0 Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-6be5jvnkdzjptdrebfn5263n@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-09-12 12:35:21 -03:00
perf trace: Add possibility to switch off syscall events Currently, we may either trace syscalls or syscalls+pagefaults. We'd like to be able to trace *only* pagefaults and this commit implements this feature. Example: [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace --no-syscalls -F -p `pidof xchat` 0.000 ( 0.000 ms): xchat/4574 majfault [g_unichar_get_script+0x11] => /usr/lib64/libglib-2.0.so.0.3800.2@0xc403b (x.) 0.202 ( 0.000 ms): xchat/4574 majfault [_cairo_hash_table_lookup+0x53] => 0x2280ff0 (?.) 20.854 ( 0.000 ms): xchat/4574 majfault [gdk_cairo_set_source_pixbuf+0x110] => /usr/bin/xchat@0x6da1f (x.) 1022.000 ( 0.000 ms): xchat/4574 majfault [__memcpy_sse2_unaligned+0x29] => 0x7ff5a8ca0400 (?.) ^C[root@zoo /]# Below we can see malloc calls, 'trace' reading symbol tables in libraries to resolve symbols, etc. [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace --no-syscalls -F all --cpu 1 sleep 10 0.000 ( 0.000 ms): chrome/26589 minfault [0x1b53129] => /tmp/perf-26589.map@0x33cbcbf7f000 (x.) 96.477 ( 0.000 ms): libvirtd/947 minfault [copy_user_enhanced_fast_string+0x5] => 0x7f7685bba000 (?k) 113.164 ( 0.000 ms): Xorg/1063 minfault [0x786da] => 0x7fce52882a3c (?.) 7162.801 ( 0.000 ms): chrome/3747 minfault [0x8e1a89] => 0xfcaefed0008 (?.) <SNIP> 7773.138 ( 0.000 ms): chrome/3886 minfault [0x8e1a89] => 0xfcb0ce28008 (?.) 7992.022 ( 0.000 ms): chrome/26574 minfault [0x1b5a708] => 0x3de7b5fc5000 (?.) 8108.949 ( 0.000 ms): qemu-system-x8/4537 majfault [_int_malloc+0xee] => 0x7faffc466d60 (?.) 8108.975 ( 0.000 ms): qemu-system-x8/4537 minfault [_int_malloc+0x102] => 0x7faffc466d60 (?.) <SNIP> 8148.174 ( 0.000 ms): qemu-system-x8/4537 minfault [_int_malloc+0x102] => 0x7faffc4eb500 (?.) 8270.855 ( 0.000 ms): chrome/26245 minfault [do_bo_emit_reloc+0xdb] => 0x45d092bc004 (?.) 8270.869 ( 0.000 ms): chrome/26245 minfault [do_bo_emit_reloc+0x108] => 0x45d09150000 (?.) no symbols found in /usr/lib64/libspice-server.so.1.9.0, maybe install a debug package? 8273.831 ( 0.000 ms): trace/20198 majfault [__memcmp_sse4_1+0xbc6] => /usr/lib64/libspice-server.so.1.9.0@0xdf000 (d.) <SNIP> 8275.121 ( 0.000 ms): trace/20198 minfault [dso__load+0x38] => 0x14fe756 (?.) no symbols found in /usr/lib64/libelf-0.158.so, maybe install a debug package? 8275.142 ( 0.000 ms): trace/20198 minfault [__memcmp_sse4_1+0xbc6] => /usr/lib64/libelf-0.158.so@0x0 (d.) <SNIP> [root@zoo /]# Signed-off-by: Stanislav Fomichev <stfomichev@yandex-team.ru> Cc: David Ahern <dsahern@gmail.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1403799268-1367-6-git-send-email-stfomichev@yandex-team.ru Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-06-26 20:14:28 +04:00
perf trace: Support using -f to override perf.data file ownership Enable perf trace to use perf.data when it is not owned by current user or root. Example: # perf trace record ls # chown Yunlong.Song:Yunlong.Song perf.data # ls -al perf.data -rw------- 1 Yunlong.Song Yunlong.Song 4153101 Apr 2 15:28 perf.data # id uid=0(root) gid=0(root) groups=0(root),64(pkcs11) Before this patch: # perf trace -i perf.data File perf.data not owned by current user or root (use -f to override) # perf trace -i perf.data -f Error: unknown switch `f' usage: perf trace [<options>] [<command>] or: perf trace [<options>] -- <command> [<options>] or: perf trace record [<options>] [<command>] or: perf trace record [<options>] -- <command> [<options>] --event <event> event selector. use 'perf list' to list available events --comm show the thread COMM next to its id --tool_stats show tool stats -e, --expr <expr> list of events to trace -o, --output <file> output file name -i, --input <file> Analyze events in file -p, --pid <pid> trace events on existing process id -t, --tid <tid> trace events on existing thread id --filter-pids <float> ... As shown above, the -f option does not work at all. After this patch: # perf trace -i perf.data File perf.data not owned by current user or root (use -f to override) # perf trace -i perf.data -f 0.056 ( 0.002 ms): ls/47325 brk( ... 0.108 ( 0.018 ms): ls/47325 mmap(len: 4096, prot: READ|WRITE, ... 0.145 ( 0.013 ms): ls/47325 access(filename: 0x7f31259a0eb0, ... 0.172 ( 0.008 ms): ls/47325 open(filename: 0x7fffeb9a0d00, ... 0.180 ( 0.004 ms): ls/47325 stat(filename: 0x7fffeb9a0d00, ... 0.185 ( 0.004 ms): ls/47325 open(filename: 0x7fffeb9a0d00, ... 0.189 ( 0.003 ms): ls/47325 stat(filename: 0x7fffeb9a0d00, ... 0.195 ( 0.004 ms): ls/47325 open(filename: 0x7fffeb9a0d00, ... 0.199 ( 0.002 ms): ls/47325 stat(filename: 0x7fffeb9a0d00, ... 0.205 ( 0.004 ms): ls/47325 open(filename: 0x7fffeb9a0d00, ... 0.211 ( 0.004 ms): ls/47325 stat(filename: 0x7fffeb9a0d00, ... 0.220 ( 0.007 ms): ls/47325 open(filename: 0x7f312599e8ff, ... ... ... As shown above, the -f option really works now. Signed-off-by: Yunlong Song <yunlong.song@huawei.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Wang Nan <wangnan0@huawei.com> Link: http://lkml.kernel.org/r/1427982439-27388-10-git-send-email-yunlong.song@huawei.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-02 21:47:18 +08:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Fix up fd -> pathname resolution There was a brown paper bag bug in the patch that introduced a reference implementation on using 'perf probe' made wannabe tracepoints that broke fd -> pathname resolution, fix it: [root@zoo ~]# perf probe 'vfs_getname=getname_flags:65 pathname=result->name:string' Added new event: probe:vfs_getname (on getname_flags:65 with pathname=result->name:string) You can now use it in all perf tools, such as: perf record -e probe:vfs_getname -aR sleep 1 [root@zoo ~] Before: [acme@zoo linux]$ trace touch -e open,fstat /tmp/b 1.159 ( 0.007 ms): open(filename: 0x7fd73f2fe088, flags: CLOEXEC ) = 3 1.163 ( 0.002 ms): fstat(fd: 3, statbuf: 0x7fff1b25e610 ) = 0 1.192 ( 0.009 ms): open(filename: 0x7fd73f4fedb8, flags: CLOEXEC ) = 3 1.201 ( 0.002 ms): fstat(fd: 3, statbuf: 0x7fff1b25e660 ) = 0 1.501 ( 0.013 ms): open(filename: 0x7fd73f0a1610, flags: CLOEXEC ) = 3 1.505 ( 0.002 ms): fstat(fd: 3, statbuf: 0x7fd73f2ddb60 ) = 0 1.581 ( 0.011 ms): open(filename: 0x7fff1b2603da, flags: CREAT|NOCTTY|NONBLOCK|WRONLY, mode: 438) = 3 [acme@zoo linux]$ After: [acme@zoo linux]$ trace touch -e open,fstat,dup2,mmap,close /tmp/b 1.105 ( 0.004 ms): mmap(len: 4096, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: -1 ) = 0x2fbf000 1.136 ( 0.008 ms): open(filename: 0x7f8902dbc088, flags: CLOEXEC ) = 3 1.140 ( 0.002 ms): fstat(fd: 3</etc/ld.so.cache>, statbuf: 0x7fff19889ef0 ) = 0 1.146 ( 0.004 ms): mmap(len: 86079, prot: READ, flags: PRIVATE, fd: 3</etc/ld.so.cache> ) = 0x2fa9000 1.149 ( 0.001 ms): close(fd: 3</etc/ld.so.cache> ) = 0 1.170 ( 0.010 ms): open(filename: 0x7f8902fbcdb8, flags: CLOEXEC ) = 3 1.178 ( 0.002 ms): fstat(fd: 3</lib64/libc.so.6>, statbuf: 0x7fff19889f40 ) = 0 1.188 ( 0.006 ms): mmap(len: 3924576, prot: EXEC|READ, flags: PRIVATE|DENYWRITE, fd: 3</lib64/libc.so.6>) = 0x29e2000 1.207 ( 0.007 ms): mmap(addr: 0x7f8902d96000, len: 24576, prot: READ|WRITE, flags: PRIVATE|DENYWRITE|FIXED, fd: 3</lib64/libc.so.6>, off: 1785856) = 0x2d96000 1.217 ( 0.004 ms): mmap(addr: 0x7f8902d9c000, len: 16992, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS|FIXED, fd: -1) = 0x2d9c000 1.228 ( 0.002 ms): close(fd: 3</lib64/libc.so.6> ) = 0 1.243 ( 0.003 ms): mmap(len: 4096, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: -1 ) = 0x2fa8000 1.250 ( 0.003 ms): mmap(len: 8192, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: -1 ) = 0x2fa6000 1.452 ( 0.010 ms): open(filename: 0x7f8902b5f610, flags: CLOEXEC ) = 3 1.455 ( 0.002 ms): fstat(fd: 3</usr/lib/locale/locale-archive>, statbuf: 0x7f8902d9bb60 ) = 0 1.461 ( 0.004 ms): mmap(len: 106070960, prot: READ, flags: PRIVATE, fd: 3</usr/lib/locale/locale-archive>) = 0xfc4b9000 1.469 ( 0.002 ms): close(fd: 3</usr/lib/locale/locale-archive> ) = 0 1.528 ( 0.010 ms): open(filename: 0x7fff1988c3da, flags: CREAT|NOCTTY|NONBLOCK|WRONLY, mode: 438) = 3 1.532 ( 0.002 ms): dup2(oldfd: 3</tmp/b> ) = 0 1.535 ( 0.001 ms): close(fd: 3</tmp/b> ) = 0 1.544 ( 0.001 ms): close( ) = 0 1.555 ( 0.001 ms): close(fd: 1 ) = 0 1.558 ( 0.001 ms): close(fd: 2 ) = 0 [acme@zoo linux]$ Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-vcm22xpjxc3j4hbyuzjzf7ik@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-06-10 16:00:18 -03:00
perf trace: Fix up fd -> pathname resolution There was a brown paper bag bug in the patch that introduced a reference implementation on using 'perf probe' made wannabe tracepoints that broke fd -> pathname resolution, fix it: [root@zoo ~]# perf probe 'vfs_getname=getname_flags:65 pathname=result->name:string' Added new event: probe:vfs_getname (on getname_flags:65 with pathname=result->name:string) You can now use it in all perf tools, such as: perf record -e probe:vfs_getname -aR sleep 1 [root@zoo ~] Before: [acme@zoo linux]$ trace touch -e open,fstat /tmp/b 1.159 ( 0.007 ms): open(filename: 0x7fd73f2fe088, flags: CLOEXEC ) = 3 1.163 ( 0.002 ms): fstat(fd: 3, statbuf: 0x7fff1b25e610 ) = 0 1.192 ( 0.009 ms): open(filename: 0x7fd73f4fedb8, flags: CLOEXEC ) = 3 1.201 ( 0.002 ms): fstat(fd: 3, statbuf: 0x7fff1b25e660 ) = 0 1.501 ( 0.013 ms): open(filename: 0x7fd73f0a1610, flags: CLOEXEC ) = 3 1.505 ( 0.002 ms): fstat(fd: 3, statbuf: 0x7fd73f2ddb60 ) = 0 1.581 ( 0.011 ms): open(filename: 0x7fff1b2603da, flags: CREAT|NOCTTY|NONBLOCK|WRONLY, mode: 438) = 3 [acme@zoo linux]$ After: [acme@zoo linux]$ trace touch -e open,fstat,dup2,mmap,close /tmp/b 1.105 ( 0.004 ms): mmap(len: 4096, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: -1 ) = 0x2fbf000 1.136 ( 0.008 ms): open(filename: 0x7f8902dbc088, flags: CLOEXEC ) = 3 1.140 ( 0.002 ms): fstat(fd: 3</etc/ld.so.cache>, statbuf: 0x7fff19889ef0 ) = 0 1.146 ( 0.004 ms): mmap(len: 86079, prot: READ, flags: PRIVATE, fd: 3</etc/ld.so.cache> ) = 0x2fa9000 1.149 ( 0.001 ms): close(fd: 3</etc/ld.so.cache> ) = 0 1.170 ( 0.010 ms): open(filename: 0x7f8902fbcdb8, flags: CLOEXEC ) = 3 1.178 ( 0.002 ms): fstat(fd: 3</lib64/libc.so.6>, statbuf: 0x7fff19889f40 ) = 0 1.188 ( 0.006 ms): mmap(len: 3924576, prot: EXEC|READ, flags: PRIVATE|DENYWRITE, fd: 3</lib64/libc.so.6>) = 0x29e2000 1.207 ( 0.007 ms): mmap(addr: 0x7f8902d96000, len: 24576, prot: READ|WRITE, flags: PRIVATE|DENYWRITE|FIXED, fd: 3</lib64/libc.so.6>, off: 1785856) = 0x2d96000 1.217 ( 0.004 ms): mmap(addr: 0x7f8902d9c000, len: 16992, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS|FIXED, fd: -1) = 0x2d9c000 1.228 ( 0.002 ms): close(fd: 3</lib64/libc.so.6> ) = 0 1.243 ( 0.003 ms): mmap(len: 4096, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: -1 ) = 0x2fa8000 1.250 ( 0.003 ms): mmap(len: 8192, prot: READ|WRITE, flags: PRIVATE|ANONYMOUS, fd: -1 ) = 0x2fa6000 1.452 ( 0.010 ms): open(filename: 0x7f8902b5f610, flags: CLOEXEC ) = 3 1.455 ( 0.002 ms): fstat(fd: 3</usr/lib/locale/locale-archive>, statbuf: 0x7f8902d9bb60 ) = 0 1.461 ( 0.004 ms): mmap(len: 106070960, prot: READ, flags: PRIVATE, fd: 3</usr/lib/locale/locale-archive>) = 0xfc4b9000 1.469 ( 0.002 ms): close(fd: 3</usr/lib/locale/locale-archive> ) = 0 1.528 ( 0.010 ms): open(filename: 0x7fff1988c3da, flags: CREAT|NOCTTY|NONBLOCK|WRONLY, mode: 438) = 3 1.532 ( 0.002 ms): dup2(oldfd: 3</tmp/b> ) = 0 1.535 ( 0.001 ms): close(fd: 3</tmp/b> ) = 0 1.544 ( 0.001 ms): close( ) = 0 1.555 ( 0.001 ms): close(fd: 1 ) = 0 1.558 ( 0.001 ms): close(fd: 2 ) = 0 [acme@zoo linux]$ Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-vcm22xpjxc3j4hbyuzjzf7ik@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-06-10 16:00:18 -03:00
perf trace: Deref sys_enter pointer args with contents from probe:vfs_getname To work like strace and dereference syscall pointer args we need to insert probes (or tracepoints) right after we copy those bytes from userspace. Since we're formatting the syscall args at raw_syscalls:sys_enter time, we need to have a formatter that just stores the position where, later, when we get the probe:vfs_getname, we can insert the pointer contents. Now, if a probe:vfs_getname with this format is in place: # perf probe -l probe:vfs_getname (on getname_flags:72@/home/git/linux/fs/namei.c with pathname) That was, in this case, put in place with: # perf probe 'vfs_getname=getname_flags:72 pathname=filename:string' Added new event: probe:vfs_getname (on getname_flags:72 with pathname=filename:string) You can now use it in all perf tools, such as: perf record -e probe:vfs_getname -aR sleep 1 # Then 'perf trace' will notice that and do the pointer -> contents expansion: # trace -e open touch /tmp/bla 0.165 (0.010 ms): touch/17752 open(filename: /etc/ld.so.cache, flags: CLOEXEC) = 3 0.195 (0.011 ms): touch/17752 open(filename: /lib64/libc.so.6, flags: CLOEXEC) = 3 0.512 (0.012 ms): touch/17752 open(filename: /usr/lib/locale/locale-archive, flags: CLOEXEC) = 3 0.582 (0.012 ms): touch/17752 open(filename: /tmp/bla, flags: CREAT|NOCTTY|NONBLOCK|WRONLY, mode: 438) = 3 # Roughly equivalent to strace's output: # strace -rT -e open touch /tmp/bla 0.000000 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 <0.000039> 0.000317 open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 <0.000102> 0.001461 open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3 <0.000072> 0.000405 open("/tmp/bla", O_WRONLY|O_CREAT|O_NOCTTY|O_NONBLOCK, 0666) = 3 <0.000055> 0.000641 +++ exited with 0 +++ # Now we need to either look for at all syscalls that are marked as pointers and have some well known names ("filename", "pathname", etc) and set the arg formatter to the one used for the "open" syscall in this patch. This implementation works for syscalls with just a string being copied from userspace, for matching syscalls with more than one string being copied via the same probe/trace point (vfs_getname) we need to extend the vfs_getname probe spec to include the pointer too, but there are some problems with that in 'perf probe' or the kernel kprobes code, need to investigate before considering supporting multiple strings per syscall. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Milian Wolff <mail@milianw.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-xvuwx6nuj8cf389kf9s2ue2s@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-08-04 22:30:09 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Add option to show process COMM Enabled by default, disable with --no-comm, e.g.: 181.821 (0.001 ms): deja-dup-monit/10784 recvmsg(fd: 8, msg: 0x7fff4342baf0, flags: PEEK|TRUNC|CMSG_CLOEXEC ) = 20 181.824 (0.001 ms): deja-dup-monit/10784 geteuid( ) = 1000 181.825 (0.001 ms): deja-dup-monit/10784 getegid( ) = 1000 181.834 (0.002 ms): deja-dup-monit/10784 recvmsg(fd: 8, msg: 0x7fff4342baf0, flags: CMSG_CLOEXEC ) = 20 181.836 (0.001 ms): deja-dup-monit/10784 geteuid( ) = 1000 181.838 (0.001 ms): deja-dup-monit/10784 getegid( ) = 1000 181.705 (0.003 ms): evolution-addr/10924 recvmsg(fd: 10, msg: 0x7fff17dc6990, flags: PEEK|TRUNC|CMSG_CLOEXEC) = 1256 181.710 (0.002 ms): evolution-addr/10924 geteuid( ) = 1000 181.712 (0.001 ms): evolution-addr/10924 getegid( ) = 1000 181.727 (0.003 ms): evolution-addr/10924 recvmsg(fd: 10, msg: 0x7fff17dc6990, flags: CMSG_CLOEXEC ) = 1256 181.731 (0.001 ms): evolution-addr/10924 geteuid( ) = 1000 181.734 (0.001 ms): evolution-addr/10924 getegid( ) = 1000 181.908 (0.002 ms): evolution-addr/10924 recvmsg(fd: 10, msg: 0x7fff17dc6990, flags: PEEK|TRUNC|CMSG_CLOEXEC) = 20 181.913 (0.001 ms): evolution-addr/10924 geteuid( ) = 1000 181.915 (0.001 ms): evolution-addr/10924 getegid( ) = 1000 181.930 (0.003 ms): evolution-addr/10924 recvmsg(fd: 10, msg: 0x7fff17dc6990, flags: CMSG_CLOEXEC ) = 20 181.934 (0.001 ms): evolution-addr/10924 geteuid( ) = 1000 181.937 (0.001 ms): evolution-addr/10924 getegid( ) = 1000 220.718 (0.010 ms): at-spi2-regist/10715 sendmsg(fd: 3, msg: 0x7fffdb8756c0, flags: NOSIGNAL ) = 200 220.741 (0.000 ms): dbus-daemon/10711 ... [continued]: epoll_wait()) = 1 220.759 (0.004 ms): dbus-daemon/10711 recvmsg(fd: 11, msg: 0x7ffff94594d0, flags: CMSG_CLOEXEC ) = 200 220.780 (0.002 ms): dbus-daemon/10711 recvmsg(fd: 11, msg: 0x7ffff94594d0, flags: CMSG_CLOEXEC ) = 200 220.788 (0.001 ms): dbus-daemon/10711 recvmsg(fd: 11, msg: 0x7ffff94594d0, flags: CMSG_CLOEXEC ) = -1 EAGAIN Resource temporarily unavailable 220.760 (0.004 ms): at-spi2-regist/10715 sendmsg(fd: 3, msg: 0x7fffdb8756c0, flags: NOSIGNAL ) = 200 220.771 (0.023 ms): perf/26347 open(filename: 0xf2e780, mode: 15918976 ) = 19 220.850 (0.002 ms): perf/26347 close(fd: 19 ) = 0 Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-6be5jvnkdzjptdrebfn5263n@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-09-12 12:35:21 -03:00
perf trace: Add option to show process COMM Enabled by default, disable with --no-comm, e.g.: 181.821 (0.001 ms): deja-dup-monit/10784 recvmsg(fd: 8, msg: 0x7fff4342baf0, flags: PEEK|TRUNC|CMSG_CLOEXEC ) = 20 181.824 (0.001 ms): deja-dup-monit/10784 geteuid( ) = 1000 181.825 (0.001 ms): deja-dup-monit/10784 getegid( ) = 1000 181.834 (0.002 ms): deja-dup-monit/10784 recvmsg(fd: 8, msg: 0x7fff4342baf0, flags: CMSG_CLOEXEC ) = 20 181.836 (0.001 ms): deja-dup-monit/10784 geteuid( ) = 1000 181.838 (0.001 ms): deja-dup-monit/10784 getegid( ) = 1000 181.705 (0.003 ms): evolution-addr/10924 recvmsg(fd: 10, msg: 0x7fff17dc6990, flags: PEEK|TRUNC|CMSG_CLOEXEC) = 1256 181.710 (0.002 ms): evolution-addr/10924 geteuid( ) = 1000 181.712 (0.001 ms): evolution-addr/10924 getegid( ) = 1000 181.727 (0.003 ms): evolution-addr/10924 recvmsg(fd: 10, msg: 0x7fff17dc6990, flags: CMSG_CLOEXEC ) = 1256 181.731 (0.001 ms): evolution-addr/10924 geteuid( ) = 1000 181.734 (0.001 ms): evolution-addr/10924 getegid( ) = 1000 181.908 (0.002 ms): evolution-addr/10924 recvmsg(fd: 10, msg: 0x7fff17dc6990, flags: PEEK|TRUNC|CMSG_CLOEXEC) = 20 181.913 (0.001 ms): evolution-addr/10924 geteuid( ) = 1000 181.915 (0.001 ms): evolution-addr/10924 getegid( ) = 1000 181.930 (0.003 ms): evolution-addr/10924 recvmsg(fd: 10, msg: 0x7fff17dc6990, flags: CMSG_CLOEXEC ) = 20 181.934 (0.001 ms): evolution-addr/10924 geteuid( ) = 1000 181.937 (0.001 ms): evolution-addr/10924 getegid( ) = 1000 220.718 (0.010 ms): at-spi2-regist/10715 sendmsg(fd: 3, msg: 0x7fffdb8756c0, flags: NOSIGNAL ) = 200 220.741 (0.000 ms): dbus-daemon/10711 ... [continued]: epoll_wait()) = 1 220.759 (0.004 ms): dbus-daemon/10711 recvmsg(fd: 11, msg: 0x7ffff94594d0, flags: CMSG_CLOEXEC ) = 200 220.780 (0.002 ms): dbus-daemon/10711 recvmsg(fd: 11, msg: 0x7ffff94594d0, flags: CMSG_CLOEXEC ) = 200 220.788 (0.001 ms): dbus-daemon/10711 recvmsg(fd: 11, msg: 0x7ffff94594d0, flags: CMSG_CLOEXEC ) = -1 EAGAIN Resource temporarily unavailable 220.760 (0.004 ms): at-spi2-regist/10715 sendmsg(fd: 3, msg: 0x7fffdb8756c0, flags: NOSIGNAL ) = 200 220.771 (0.023 ms): perf/26347 open(filename: 0xf2e780, mode: 15918976 ) = 19 220.850 (0.002 ms): perf/26347 close(fd: 19 ) = 0 Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-6be5jvnkdzjptdrebfn5263n@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-09-12 12:35:21 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf tools: Check recorded kernel version when finding vmlinux Currently vmlinux_path__init() only tries to find vmlinux file from current directory, /boot and some canonical directories with version number of the running kernel. This can be a problem when reporting old data recorded on a kernel version not running currently. We can use --symfs option for this but it's annoying for user to do it always. As we already have the info in the perf.data file, it can be changed to use it for the search automatically. Before: $ perf report ... # Samples: 4K of event 'cpu-clock' # Event count (approx.): 1067250000 # # Overhead Command Shared Object Symbol # ........ .......... ................. .............................. 71.87% swapper [kernel.kallsyms] [k] recover_probed_instruction After: # Overhead Command Shared Object Symbol # ........ .......... ................. .................... 71.87% swapper [kernel.kallsyms] [k] native_safe_halt This requires to change signature of symbol__init() to receive struct perf_session_env *. Reported-by: Minchan Kim <minchan@kernel.org> Signed-off-by: Namhyung Kim <namhyung@kernel.org> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com> Cc: Minchan Kim <minchan@kernel.org> Cc: Namhyung Kim <namhyung.kim@lge.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1407825645-24586-14-git-send-email-namhyung@kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-08-12 15:40:45 +09:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf python: Remove dependency on 'machine' methods The python binding still doesn't provide symbol resolving facilities, but the recent addition of the trace_event__register_resolver() function made it add as a dependency the machine__resolve_kernel_addr() method, that in turn drags all the symbol resolving code. The problem: [root@zoo ~]# perf test -v python 17: Try 'import perf' in python, checking link problems : --- start --- test child forked, pid 6853 Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: /tmp/build/perf/python/perf.so: undefined symbol: machine__resolve_kernel_addr test child finished with -1 ---- end ---- Try 'import perf' in python, checking link problems: FAILED! [root@zoo ~]# Fix it by requiring this function to receive the resolver as a parameter, just like pevent_register_function_resolver(), i.e. do not explicitely refer to an object file not included in tools/perf/util/python-ext-sources. [root@zoo ~]# perf test python 17: Try 'import perf' in python, checking link problems : Ok [root@zoo ~]# Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Fixes: c3168b0db93a ("perf symbols: Provide libtraceevent callback to resolve kernel symbols") Link: http://lkml.kernel.org/n/tip-vxlhh95v2em9zdbgj3jm7xi5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-07-24 12:13:05 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Fix SIGBUS failures due to misaligned accesses On Sparc64 perf-trace is failing in many spots due to extended load instructions being used on misaligned accesses. (gdb) run trace ls Starting program: /tmp/perf/perf trace ls [Thread debugging using libthread_db enabled] Detaching after fork from child process 169460. <ls output removed> Program received signal SIGBUS, Bus error. 0x000000000014f4dc in tp_field__u64 (field=0x4cc700, sample=0x7feffffa098) at builtin-trace.c:61 warning: Source file is more recent than executable. 61 TP_UINT_FIELD(64); (gdb) bt 0 0x000000000014f4dc in tp_field__u64 (field=0x4cc700, sample=0x7feffffa098) at builtin-trace.c:61 1 0x0000000000156ad4 in trace__sys_exit (trace=0x7feffffc268, evsel=0x4cc580, event=0xfffffc0104912000, sample=0x7feffffa098) at builtin-trace.c:1701 2 0x0000000000158c14 in trace__run (trace=0x7feffffc268, argc=1, argv=0x7fefffff360) at builtin-trace.c:2160 3 0x000000000015b78c in cmd_trace (argc=1, argv=0x7fefffff360, prefix=0x0) at builtin-trace.c:2609 4 0x0000000000107d94 in run_builtin (p=0x4549c8, argc=2, argv=0x7fefffff360) at perf.c:341 5 0x0000000000108140 in handle_internal_command (argc=2, argv=0x7fefffff360) at perf.c:400 6 0x0000000000108308 in run_argv (argcp=0x7feffffef2c, argv=0x7feffffef20) at perf.c:444 7 0x0000000000108728 in main (argc=2, argv=0x7fefffff360) at perf.c:559 (gdb) p *sample $1 = {ip = 4391276, pid = 169472, tid = 169472, time = 6303014583281250, addr = 0, id = 72082, stream_id = 18446744073709551615, period = 1, weight = 0, transaction = 0, cpu = 73, raw_size = 36, data_src = 84410401, flags = 0, insn_len = 0, raw_data = 0xfffffc010491203c, callchain = 0x0, branch_stack = 0x0, user_regs = {abi = 0, mask = 0, regs = 0x0, cache_regs = 0x7feffffa098, cache_mask = 0}, intr_regs = {abi = 0, mask = 0, regs = 0x0, cache_regs = 0x7feffffa098, cache_mask = 0}, user_stack = { offset = 0, size = 0, data = 0x0}, read = {time_enabled = 0, time_running = 0, {group = {nr = 0, values = 0x0}, one = {value = 0, id = 0}}}} (gdb) p *field $2 = {offset = 16, {integer = 0x14f4a8 <tp_field__u64>, pointer = 0x14f4a8 <tp_field__u64>}} sample->raw_data is guaranteed to not be 8-byte aligned because it is preceded by the size as a u3. So accessing raw data with an extended load instruction causes the SIGBUS. Resolve by using memcpy to a temporary variable of appropriate size. Signed-off-by: David Ahern <david.ahern@oracle.com> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1424376022-140608-1-git-send-email-david.ahern@oracle.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-19 15:00:22 -05:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Fix SIGBUS failures due to misaligned accesses On Sparc64 perf-trace is failing in many spots due to extended load instructions being used on misaligned accesses. (gdb) run trace ls Starting program: /tmp/perf/perf trace ls [Thread debugging using libthread_db enabled] Detaching after fork from child process 169460. <ls output removed> Program received signal SIGBUS, Bus error. 0x000000000014f4dc in tp_field__u64 (field=0x4cc700, sample=0x7feffffa098) at builtin-trace.c:61 warning: Source file is more recent than executable. 61 TP_UINT_FIELD(64); (gdb) bt 0 0x000000000014f4dc in tp_field__u64 (field=0x4cc700, sample=0x7feffffa098) at builtin-trace.c:61 1 0x0000000000156ad4 in trace__sys_exit (trace=0x7feffffc268, evsel=0x4cc580, event=0xfffffc0104912000, sample=0x7feffffa098) at builtin-trace.c:1701 2 0x0000000000158c14 in trace__run (trace=0x7feffffc268, argc=1, argv=0x7fefffff360) at builtin-trace.c:2160 3 0x000000000015b78c in cmd_trace (argc=1, argv=0x7fefffff360, prefix=0x0) at builtin-trace.c:2609 4 0x0000000000107d94 in run_builtin (p=0x4549c8, argc=2, argv=0x7fefffff360) at perf.c:341 5 0x0000000000108140 in handle_internal_command (argc=2, argv=0x7fefffff360) at perf.c:400 6 0x0000000000108308 in run_argv (argcp=0x7feffffef2c, argv=0x7feffffef20) at perf.c:444 7 0x0000000000108728 in main (argc=2, argv=0x7fefffff360) at perf.c:559 (gdb) p *sample $1 = {ip = 4391276, pid = 169472, tid = 169472, time = 6303014583281250, addr = 0, id = 72082, stream_id = 18446744073709551615, period = 1, weight = 0, transaction = 0, cpu = 73, raw_size = 36, data_src = 84410401, flags = 0, insn_len = 0, raw_data = 0xfffffc010491203c, callchain = 0x0, branch_stack = 0x0, user_regs = {abi = 0, mask = 0, regs = 0x0, cache_regs = 0x7feffffa098, cache_mask = 0}, intr_regs = {abi = 0, mask = 0, regs = 0x0, cache_regs = 0x7feffffa098, cache_mask = 0}, user_stack = { offset = 0, size = 0, data = 0x0}, read = {time_enabled = 0, time_running = 0, {group = {nr = 0, values = 0x0}, one = {value = 0, id = 0}}}} (gdb) p *field $2 = {offset = 16, {integer = 0x14f4a8 <tp_field__u64>, pointer = 0x14f4a8 <tp_field__u64>}} sample->raw_data is guaranteed to not be 8-byte aligned because it is preceded by the size as a u3. So accessing raw data with an extended load instruction causes the SIGBUS. Resolve by using memcpy to a temporary variable of appropriate size. Signed-off-by: David Ahern <david.ahern@oracle.com> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1424376022-140608-1-git-send-email-david.ahern@oracle.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-19 15:00:22 -05:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Fix SIGBUS failures due to misaligned accesses On Sparc64 perf-trace is failing in many spots due to extended load instructions being used on misaligned accesses. (gdb) run trace ls Starting program: /tmp/perf/perf trace ls [Thread debugging using libthread_db enabled] Detaching after fork from child process 169460. <ls output removed> Program received signal SIGBUS, Bus error. 0x000000000014f4dc in tp_field__u64 (field=0x4cc700, sample=0x7feffffa098) at builtin-trace.c:61 warning: Source file is more recent than executable. 61 TP_UINT_FIELD(64); (gdb) bt 0 0x000000000014f4dc in tp_field__u64 (field=0x4cc700, sample=0x7feffffa098) at builtin-trace.c:61 1 0x0000000000156ad4 in trace__sys_exit (trace=0x7feffffc268, evsel=0x4cc580, event=0xfffffc0104912000, sample=0x7feffffa098) at builtin-trace.c:1701 2 0x0000000000158c14 in trace__run (trace=0x7feffffc268, argc=1, argv=0x7fefffff360) at builtin-trace.c:2160 3 0x000000000015b78c in cmd_trace (argc=1, argv=0x7fefffff360, prefix=0x0) at builtin-trace.c:2609 4 0x0000000000107d94 in run_builtin (p=0x4549c8, argc=2, argv=0x7fefffff360) at perf.c:341 5 0x0000000000108140 in handle_internal_command (argc=2, argv=0x7fefffff360) at perf.c:400 6 0x0000000000108308 in run_argv (argcp=0x7feffffef2c, argv=0x7feffffef20) at perf.c:444 7 0x0000000000108728 in main (argc=2, argv=0x7fefffff360) at perf.c:559 (gdb) p *sample $1 = {ip = 4391276, pid = 169472, tid = 169472, time = 6303014583281250, addr = 0, id = 72082, stream_id = 18446744073709551615, period = 1, weight = 0, transaction = 0, cpu = 73, raw_size = 36, data_src = 84410401, flags = 0, insn_len = 0, raw_data = 0xfffffc010491203c, callchain = 0x0, branch_stack = 0x0, user_regs = {abi = 0, mask = 0, regs = 0x0, cache_regs = 0x7feffffa098, cache_mask = 0}, intr_regs = {abi = 0, mask = 0, regs = 0x0, cache_regs = 0x7feffffa098, cache_mask = 0}, user_stack = { offset = 0, size = 0, data = 0x0}, read = {time_enabled = 0, time_running = 0, {group = {nr = 0, values = 0x0}, one = {value = 0, id = 0}}}} (gdb) p *field $2 = {offset = 16, {integer = 0x14f4a8 <tp_field__u64>, pointer = 0x14f4a8 <tp_field__u64>}} sample->raw_data is guaranteed to not be 8-byte aligned because it is preceded by the size as a u3. So accessing raw data with an extended load instruction causes the SIGBUS. Resolve by using memcpy to a temporary variable of appropriate size. Signed-off-by: David Ahern <david.ahern@oracle.com> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1424376022-140608-1-git-send-email-david.ahern@oracle.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-19 15:00:22 -05:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Fix SIGBUS failures due to misaligned accesses On Sparc64 perf-trace is failing in many spots due to extended load instructions being used on misaligned accesses. (gdb) run trace ls Starting program: /tmp/perf/perf trace ls [Thread debugging using libthread_db enabled] Detaching after fork from child process 169460. <ls output removed> Program received signal SIGBUS, Bus error. 0x000000000014f4dc in tp_field__u64 (field=0x4cc700, sample=0x7feffffa098) at builtin-trace.c:61 warning: Source file is more recent than executable. 61 TP_UINT_FIELD(64); (gdb) bt 0 0x000000000014f4dc in tp_field__u64 (field=0x4cc700, sample=0x7feffffa098) at builtin-trace.c:61 1 0x0000000000156ad4 in trace__sys_exit (trace=0x7feffffc268, evsel=0x4cc580, event=0xfffffc0104912000, sample=0x7feffffa098) at builtin-trace.c:1701 2 0x0000000000158c14 in trace__run (trace=0x7feffffc268, argc=1, argv=0x7fefffff360) at builtin-trace.c:2160 3 0x000000000015b78c in cmd_trace (argc=1, argv=0x7fefffff360, prefix=0x0) at builtin-trace.c:2609 4 0x0000000000107d94 in run_builtin (p=0x4549c8, argc=2, argv=0x7fefffff360) at perf.c:341 5 0x0000000000108140 in handle_internal_command (argc=2, argv=0x7fefffff360) at perf.c:400 6 0x0000000000108308 in run_argv (argcp=0x7feffffef2c, argv=0x7feffffef20) at perf.c:444 7 0x0000000000108728 in main (argc=2, argv=0x7fefffff360) at perf.c:559 (gdb) p *sample $1 = {ip = 4391276, pid = 169472, tid = 169472, time = 6303014583281250, addr = 0, id = 72082, stream_id = 18446744073709551615, period = 1, weight = 0, transaction = 0, cpu = 73, raw_size = 36, data_src = 84410401, flags = 0, insn_len = 0, raw_data = 0xfffffc010491203c, callchain = 0x0, branch_stack = 0x0, user_regs = {abi = 0, mask = 0, regs = 0x0, cache_regs = 0x7feffffa098, cache_mask = 0}, intr_regs = {abi = 0, mask = 0, regs = 0x0, cache_regs = 0x7feffffa098, cache_mask = 0}, user_stack = { offset = 0, size = 0, data = 0x0}, read = {time_enabled = 0, time_running = 0, {group = {nr = 0, values = 0x0}, one = {value = 0, id = 0}}}} (gdb) p *field $2 = {offset = 16, {integer = 0x14f4a8 <tp_field__u64>, pointer = 0x14f4a8 <tp_field__u64>}} sample->raw_data is guaranteed to not be 8-byte aligned because it is preceded by the size as a u3. So accessing raw data with an extended load instruction causes the SIGBUS. Resolve by using memcpy to a temporary variable of appropriate size. Signed-off-by: David Ahern <david.ahern@oracle.com> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1424376022-140608-1-git-send-email-david.ahern@oracle.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-19 15:00:22 -05:00
perf trace: Fix SIGBUS failures due to misaligned accesses On Sparc64 perf-trace is failing in many spots due to extended load instructions being used on misaligned accesses. (gdb) run trace ls Starting program: /tmp/perf/perf trace ls [Thread debugging using libthread_db enabled] Detaching after fork from child process 169460. <ls output removed> Program received signal SIGBUS, Bus error. 0x000000000014f4dc in tp_field__u64 (field=0x4cc700, sample=0x7feffffa098) at builtin-trace.c:61 warning: Source file is more recent than executable. 61 TP_UINT_FIELD(64); (gdb) bt 0 0x000000000014f4dc in tp_field__u64 (field=0x4cc700, sample=0x7feffffa098) at builtin-trace.c:61 1 0x0000000000156ad4 in trace__sys_exit (trace=0x7feffffc268, evsel=0x4cc580, event=0xfffffc0104912000, sample=0x7feffffa098) at builtin-trace.c:1701 2 0x0000000000158c14 in trace__run (trace=0x7feffffc268, argc=1, argv=0x7fefffff360) at builtin-trace.c:2160 3 0x000000000015b78c in cmd_trace (argc=1, argv=0x7fefffff360, prefix=0x0) at builtin-trace.c:2609 4 0x0000000000107d94 in run_builtin (p=0x4549c8, argc=2, argv=0x7fefffff360) at perf.c:341 5 0x0000000000108140 in handle_internal_command (argc=2, argv=0x7fefffff360) at perf.c:400 6 0x0000000000108308 in run_argv (argcp=0x7feffffef2c, argv=0x7feffffef20) at perf.c:444 7 0x0000000000108728 in main (argc=2, argv=0x7fefffff360) at perf.c:559 (gdb) p *sample $1 = {ip = 4391276, pid = 169472, tid = 169472, time = 6303014583281250, addr = 0, id = 72082, stream_id = 18446744073709551615, period = 1, weight = 0, transaction = 0, cpu = 73, raw_size = 36, data_src = 84410401, flags = 0, insn_len = 0, raw_data = 0xfffffc010491203c, callchain = 0x0, branch_stack = 0x0, user_regs = {abi = 0, mask = 0, regs = 0x0, cache_regs = 0x7feffffa098, cache_mask = 0}, intr_regs = {abi = 0, mask = 0, regs = 0x0, cache_regs = 0x7feffffa098, cache_mask = 0}, user_stack = { offset = 0, size = 0, data = 0x0}, read = {time_enabled = 0, time_running = 0, {group = {nr = 0, values = 0x0}, one = {value = 0, id = 0}}}} (gdb) p *field $2 = {offset = 16, {integer = 0x14f4a8 <tp_field__u64>, pointer = 0x14f4a8 <tp_field__u64>}} sample->raw_data is guaranteed to not be 8-byte aligned because it is preceded by the size as a u3. So accessing raw data with an extended load instruction causes the SIGBUS. Resolve by using memcpy to a temporary variable of appropriate size. Signed-off-by: David Ahern <david.ahern@oracle.com> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1424376022-140608-1-git-send-email-david.ahern@oracle.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-19 15:00:22 -05:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Fix SIGBUS failures due to misaligned accesses On Sparc64 perf-trace is failing in many spots due to extended load instructions being used on misaligned accesses. (gdb) run trace ls Starting program: /tmp/perf/perf trace ls [Thread debugging using libthread_db enabled] Detaching after fork from child process 169460. <ls output removed> Program received signal SIGBUS, Bus error. 0x000000000014f4dc in tp_field__u64 (field=0x4cc700, sample=0x7feffffa098) at builtin-trace.c:61 warning: Source file is more recent than executable. 61 TP_UINT_FIELD(64); (gdb) bt 0 0x000000000014f4dc in tp_field__u64 (field=0x4cc700, sample=0x7feffffa098) at builtin-trace.c:61 1 0x0000000000156ad4 in trace__sys_exit (trace=0x7feffffc268, evsel=0x4cc580, event=0xfffffc0104912000, sample=0x7feffffa098) at builtin-trace.c:1701 2 0x0000000000158c14 in trace__run (trace=0x7feffffc268, argc=1, argv=0x7fefffff360) at builtin-trace.c:2160 3 0x000000000015b78c in cmd_trace (argc=1, argv=0x7fefffff360, prefix=0x0) at builtin-trace.c:2609 4 0x0000000000107d94 in run_builtin (p=0x4549c8, argc=2, argv=0x7fefffff360) at perf.c:341 5 0x0000000000108140 in handle_internal_command (argc=2, argv=0x7fefffff360) at perf.c:400 6 0x0000000000108308 in run_argv (argcp=0x7feffffef2c, argv=0x7feffffef20) at perf.c:444 7 0x0000000000108728 in main (argc=2, argv=0x7fefffff360) at perf.c:559 (gdb) p *sample $1 = {ip = 4391276, pid = 169472, tid = 169472, time = 6303014583281250, addr = 0, id = 72082, stream_id = 18446744073709551615, period = 1, weight = 0, transaction = 0, cpu = 73, raw_size = 36, data_src = 84410401, flags = 0, insn_len = 0, raw_data = 0xfffffc010491203c, callchain = 0x0, branch_stack = 0x0, user_regs = {abi = 0, mask = 0, regs = 0x0, cache_regs = 0x7feffffa098, cache_mask = 0}, intr_regs = {abi = 0, mask = 0, regs = 0x0, cache_regs = 0x7feffffa098, cache_mask = 0}, user_stack = { offset = 0, size = 0, data = 0x0}, read = {time_enabled = 0, time_running = 0, {group = {nr = 0, values = 0x0}, one = {value = 0, id = 0}}}} (gdb) p *field $2 = {offset = 16, {integer = 0x14f4a8 <tp_field__u64>, pointer = 0x14f4a8 <tp_field__u64>}} sample->raw_data is guaranteed to not be 8-byte aligned because it is preceded by the size as a u3. So accessing raw data with an extended load instruction causes the SIGBUS. Resolve by using memcpy to a temporary variable of appropriate size. Signed-off-by: David Ahern <david.ahern@oracle.com> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1424376022-140608-1-git-send-email-david.ahern@oracle.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-19 15:00:22 -05:00
perf trace: Fix SIGBUS failures due to misaligned accesses On Sparc64 perf-trace is failing in many spots due to extended load instructions being used on misaligned accesses. (gdb) run trace ls Starting program: /tmp/perf/perf trace ls [Thread debugging using libthread_db enabled] Detaching after fork from child process 169460. <ls output removed> Program received signal SIGBUS, Bus error. 0x000000000014f4dc in tp_field__u64 (field=0x4cc700, sample=0x7feffffa098) at builtin-trace.c:61 warning: Source file is more recent than executable. 61 TP_UINT_FIELD(64); (gdb) bt 0 0x000000000014f4dc in tp_field__u64 (field=0x4cc700, sample=0x7feffffa098) at builtin-trace.c:61 1 0x0000000000156ad4 in trace__sys_exit (trace=0x7feffffc268, evsel=0x4cc580, event=0xfffffc0104912000, sample=0x7feffffa098) at builtin-trace.c:1701 2 0x0000000000158c14 in trace__run (trace=0x7feffffc268, argc=1, argv=0x7fefffff360) at builtin-trace.c:2160 3 0x000000000015b78c in cmd_trace (argc=1, argv=0x7fefffff360, prefix=0x0) at builtin-trace.c:2609 4 0x0000000000107d94 in run_builtin (p=0x4549c8, argc=2, argv=0x7fefffff360) at perf.c:341 5 0x0000000000108140 in handle_internal_command (argc=2, argv=0x7fefffff360) at perf.c:400 6 0x0000000000108308 in run_argv (argcp=0x7feffffef2c, argv=0x7feffffef20) at perf.c:444 7 0x0000000000108728 in main (argc=2, argv=0x7fefffff360) at perf.c:559 (gdb) p *sample $1 = {ip = 4391276, pid = 169472, tid = 169472, time = 6303014583281250, addr = 0, id = 72082, stream_id = 18446744073709551615, period = 1, weight = 0, transaction = 0, cpu = 73, raw_size = 36, data_src = 84410401, flags = 0, insn_len = 0, raw_data = 0xfffffc010491203c, callchain = 0x0, branch_stack = 0x0, user_regs = {abi = 0, mask = 0, regs = 0x0, cache_regs = 0x7feffffa098, cache_mask = 0}, intr_regs = {abi = 0, mask = 0, regs = 0x0, cache_regs = 0x7feffffa098, cache_mask = 0}, user_stack = { offset = 0, size = 0, data = 0x0}, read = {time_enabled = 0, time_running = 0, {group = {nr = 0, values = 0x0}, one = {value = 0, id = 0}}}} (gdb) p *field $2 = {offset = 16, {integer = 0x14f4a8 <tp_field__u64>, pointer = 0x14f4a8 <tp_field__u64>}} sample->raw_data is guaranteed to not be 8-byte aligned because it is preceded by the size as a u3. So accessing raw data with an extended load instruction causes the SIGBUS. Resolve by using memcpy to a temporary variable of appropriate size. Signed-off-by: David Ahern <david.ahern@oracle.com> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1424376022-140608-1-git-send-email-david.ahern@oracle.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-19 15:00:22 -05:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Fix SIGBUS failures due to misaligned accesses On Sparc64 perf-trace is failing in many spots due to extended load instructions being used on misaligned accesses. (gdb) run trace ls Starting program: /tmp/perf/perf trace ls [Thread debugging using libthread_db enabled] Detaching after fork from child process 169460. <ls output removed> Program received signal SIGBUS, Bus error. 0x000000000014f4dc in tp_field__u64 (field=0x4cc700, sample=0x7feffffa098) at builtin-trace.c:61 warning: Source file is more recent than executable. 61 TP_UINT_FIELD(64); (gdb) bt 0 0x000000000014f4dc in tp_field__u64 (field=0x4cc700, sample=0x7feffffa098) at builtin-trace.c:61 1 0x0000000000156ad4 in trace__sys_exit (trace=0x7feffffc268, evsel=0x4cc580, event=0xfffffc0104912000, sample=0x7feffffa098) at builtin-trace.c:1701 2 0x0000000000158c14 in trace__run (trace=0x7feffffc268, argc=1, argv=0x7fefffff360) at builtin-trace.c:2160 3 0x000000000015b78c in cmd_trace (argc=1, argv=0x7fefffff360, prefix=0x0) at builtin-trace.c:2609 4 0x0000000000107d94 in run_builtin (p=0x4549c8, argc=2, argv=0x7fefffff360) at perf.c:341 5 0x0000000000108140 in handle_internal_command (argc=2, argv=0x7fefffff360) at perf.c:400 6 0x0000000000108308 in run_argv (argcp=0x7feffffef2c, argv=0x7feffffef20) at perf.c:444 7 0x0000000000108728 in main (argc=2, argv=0x7fefffff360) at perf.c:559 (gdb) p *sample $1 = {ip = 4391276, pid = 169472, tid = 169472, time = 6303014583281250, addr = 0, id = 72082, stream_id = 18446744073709551615, period = 1, weight = 0, transaction = 0, cpu = 73, raw_size = 36, data_src = 84410401, flags = 0, insn_len = 0, raw_data = 0xfffffc010491203c, callchain = 0x0, branch_stack = 0x0, user_regs = {abi = 0, mask = 0, regs = 0x0, cache_regs = 0x7feffffa098, cache_mask = 0}, intr_regs = {abi = 0, mask = 0, regs = 0x0, cache_regs = 0x7feffffa098, cache_mask = 0}, user_stack = { offset = 0, size = 0, data = 0x0}, read = {time_enabled = 0, time_running = 0, {group = {nr = 0, values = 0x0}, one = {value = 0, id = 0}}}} (gdb) p *field $2 = {offset = 16, {integer = 0x14f4a8 <tp_field__u64>, pointer = 0x14f4a8 <tp_field__u64>}} sample->raw_data is guaranteed to not be 8-byte aligned because it is preceded by the size as a u3. So accessing raw data with an extended load instruction causes the SIGBUS. Resolve by using memcpy to a temporary variable of appropriate size. Signed-off-by: David Ahern <david.ahern@oracle.com> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1424376022-140608-1-git-send-email-david.ahern@oracle.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-19 15:00:22 -05:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Fix SIGBUS failures due to misaligned accesses On Sparc64 perf-trace is failing in many spots due to extended load instructions being used on misaligned accesses. (gdb) run trace ls Starting program: /tmp/perf/perf trace ls [Thread debugging using libthread_db enabled] Detaching after fork from child process 169460. <ls output removed> Program received signal SIGBUS, Bus error. 0x000000000014f4dc in tp_field__u64 (field=0x4cc700, sample=0x7feffffa098) at builtin-trace.c:61 warning: Source file is more recent than executable. 61 TP_UINT_FIELD(64); (gdb) bt 0 0x000000000014f4dc in tp_field__u64 (field=0x4cc700, sample=0x7feffffa098) at builtin-trace.c:61 1 0x0000000000156ad4 in trace__sys_exit (trace=0x7feffffc268, evsel=0x4cc580, event=0xfffffc0104912000, sample=0x7feffffa098) at builtin-trace.c:1701 2 0x0000000000158c14 in trace__run (trace=0x7feffffc268, argc=1, argv=0x7fefffff360) at builtin-trace.c:2160 3 0x000000000015b78c in cmd_trace (argc=1, argv=0x7fefffff360, prefix=0x0) at builtin-trace.c:2609 4 0x0000000000107d94 in run_builtin (p=0x4549c8, argc=2, argv=0x7fefffff360) at perf.c:341 5 0x0000000000108140 in handle_internal_command (argc=2, argv=0x7fefffff360) at perf.c:400 6 0x0000000000108308 in run_argv (argcp=0x7feffffef2c, argv=0x7feffffef20) at perf.c:444 7 0x0000000000108728 in main (argc=2, argv=0x7fefffff360) at perf.c:559 (gdb) p *sample $1 = {ip = 4391276, pid = 169472, tid = 169472, time = 6303014583281250, addr = 0, id = 72082, stream_id = 18446744073709551615, period = 1, weight = 0, transaction = 0, cpu = 73, raw_size = 36, data_src = 84410401, flags = 0, insn_len = 0, raw_data = 0xfffffc010491203c, callchain = 0x0, branch_stack = 0x0, user_regs = {abi = 0, mask = 0, regs = 0x0, cache_regs = 0x7feffffa098, cache_mask = 0}, intr_regs = {abi = 0, mask = 0, regs = 0x0, cache_regs = 0x7feffffa098, cache_mask = 0}, user_stack = { offset = 0, size = 0, data = 0x0}, read = {time_enabled = 0, time_running = 0, {group = {nr = 0, values = 0x0}, one = {value = 0, id = 0}}}} (gdb) p *field $2 = {offset = 16, {integer = 0x14f4a8 <tp_field__u64>, pointer = 0x14f4a8 <tp_field__u64>}} sample->raw_data is guaranteed to not be 8-byte aligned because it is preceded by the size as a u3. So accessing raw data with an extended load instruction causes the SIGBUS. Resolve by using memcpy to a temporary variable of appropriate size. Signed-off-by: David Ahern <david.ahern@oracle.com> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1424376022-140608-1-git-send-email-david.ahern@oracle.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-19 15:00:22 -05:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Add summary option to dump syscall statistics When enabled dumps a summary of all syscalls by task with the usual statistics -- min, max, average and relative stddev. For example, make - 26341 : 3344 [ 17.4% ] 0.000 ms read : 52 0.000 4.802 0.644 30.08 write : 20 0.004 0.036 0.010 21.72 open : 24 0.003 0.046 0.014 23.68 close : 64 0.002 0.055 0.008 22.53 stat : 2714 0.002 0.222 0.004 4.47 fstat : 18 0.001 0.041 0.006 46.26 mmap : 30 0.003 0.009 0.006 5.71 mprotect : 8 0.006 0.039 0.016 32.16 munmap : 12 0.007 0.077 0.020 38.25 brk : 48 0.002 0.014 0.004 10.18 rt_sigaction : 18 0.002 0.002 0.002 2.11 rt_sigprocmask : 60 0.002 0.128 0.010 32.88 access : 2 0.006 0.006 0.006 0.00 pipe : 12 0.004 0.048 0.013 35.98 vfork : 34 0.448 0.980 0.692 3.04 execve : 20 0.000 0.387 0.046 56.66 wait4 : 34 0.017 9923.287 593.221 68.45 fcntl : 8 0.001 0.041 0.013 48.79 getdents : 48 0.002 0.079 0.013 19.62 getcwd : 2 0.005 0.005 0.005 0.00 chdir : 2 0.070 0.070 0.070 0.00 getrlimit : 2 0.045 0.045 0.045 0.00 arch_prctl : 2 0.002 0.002 0.002 0.00 setrlimit : 2 0.002 0.002 0.002 0.00 openat : 94 0.003 0.005 0.003 2.11 Signed-off-by: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung.kim@lge.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1381289214-24885-3-git-send-email-dsahern@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-08 21:26:53 -06:00
perf trace: Add summary option to dump syscall statistics When enabled dumps a summary of all syscalls by task with the usual statistics -- min, max, average and relative stddev. For example, make - 26341 : 3344 [ 17.4% ] 0.000 ms read : 52 0.000 4.802 0.644 30.08 write : 20 0.004 0.036 0.010 21.72 open : 24 0.003 0.046 0.014 23.68 close : 64 0.002 0.055 0.008 22.53 stat : 2714 0.002 0.222 0.004 4.47 fstat : 18 0.001 0.041 0.006 46.26 mmap : 30 0.003 0.009 0.006 5.71 mprotect : 8 0.006 0.039 0.016 32.16 munmap : 12 0.007 0.077 0.020 38.25 brk : 48 0.002 0.014 0.004 10.18 rt_sigaction : 18 0.002 0.002 0.002 2.11 rt_sigprocmask : 60 0.002 0.128 0.010 32.88 access : 2 0.006 0.006 0.006 0.00 pipe : 12 0.004 0.048 0.013 35.98 vfork : 34 0.448 0.980 0.692 3.04 execve : 20 0.000 0.387 0.046 56.66 wait4 : 34 0.017 9923.287 593.221 68.45 fcntl : 8 0.001 0.041 0.013 48.79 getdents : 48 0.002 0.079 0.013 19.62 getcwd : 2 0.005 0.005 0.005 0.00 chdir : 2 0.070 0.070 0.070 0.00 getrlimit : 2 0.045 0.045 0.045 0.00 arch_prctl : 2 0.002 0.002 0.002 0.00 setrlimit : 2 0.002 0.002 0.002 0.00 openat : 94 0.003 0.005 0.003 2.11 Signed-off-by: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung.kim@lge.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1381289214-24885-3-git-send-email-dsahern@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-08 21:26:53 -06:00
perf trace: Handle multiple threads better wrt syscalls being intermixed $ trace time taskset -c 0 usleep 1 0.845 ( 0.021 ms): time/16722 wait4(upid: 4294967295, stat_addr: 0x7fff17f443d4, ru: 0x7fff17f44438 ) ... 0.865 ( 0.008 ms): time/16723 execve(arg0: 140733595272004, arg1: 140733595272720, arg2: 140733595272768, arg3: 139755107218496, arg4: 7307199665339051828, arg5: 3) = -2 2.395 ( 1.523 ms): taskset/16723 execve(arg0: 140733595272013, arg1: 140733595272720, arg2: 140733595272768, arg3: 139755107218496, arg4: 7307199665339051828, arg5: 3) = 0 2.411 ( 0.002 ms): taskset/16723 brk( ) = 0x1915000 3.300 ( 0.058 ms): usleep/16723 nanosleep(rqtp: 0x7ffff4ada190 ) = 0 <SNIP> 3.305 ( 0.000 ms): usleep/16723 exit_group( 3.363 ( 2.539 ms): time/16722 ... [continued]: wait4()) = 16723 3.366 ( 0.001 ms): time/16722 rt_sigaction(sig: INT, act: 0x7fff17f44160, oact: 0x7fff17f44200, sigsetsize: 8) = 0 We we're not seeing this line: 0.845 ( 0.021 ms): time/16722 wait4(upid: 4294967295, stat_addr: 0x7fff17f443d4, ru: 0x7fff17f44438 ) ... just the one when it finishes: 3.363 ( 2.539 ms): time/16722 ... [continued]: wait4()) = 16723 Still some issues left till we move to ordered_samples when multiple CPUs/threads are involved... Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-zq9x30a1ky3djqewqn2v3ja3@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-13 13:22:21 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf machine: Protect the machine->threads with a rwlock In addition to using refcounts for the struct thread lifetime management, we need to protect access to machine->threads from concurrent access. That happens in 'perf top', where a thread processes events, inserting and deleting entries from that rb_tree while another thread decays hist_entries, that end up dropping references and ultimately deleting threads from the rb_tree and releasing its resources when no further hist_entry (or other data structures, like in 'perf sched') references it. So the rule is the same for refcounts + protected trees in the kernel, get the tree lock, find object, bump the refcount, drop the tree lock, return, use object, drop the refcount if no more use of it is needed, keep it if storing it in some other data structure, drop when releasing that data structure. I.e. pair "t = machine__find(new)_thread()" with a "thread__put(t)", and "perf_event__preprocess_sample(&al)" with "addr_location__put(&al)". The addr_location__put() one is because as we return references to several data structures, we may end up adding more reference counting for the other data structures and then we'll drop it at addr_location__put() time. Acked-by: David Ahern <dsahern@gmail.com> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-bs9rt4n0jw3hi9f3zxyy3xln@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-06 20:43:22 -03:00
perf trace: Add summary option to dump syscall statistics When enabled dumps a summary of all syscalls by task with the usual statistics -- min, max, average and relative stddev. For example, make - 26341 : 3344 [ 17.4% ] 0.000 ms read : 52 0.000 4.802 0.644 30.08 write : 20 0.004 0.036 0.010 21.72 open : 24 0.003 0.046 0.014 23.68 close : 64 0.002 0.055 0.008 22.53 stat : 2714 0.002 0.222 0.004 4.47 fstat : 18 0.001 0.041 0.006 46.26 mmap : 30 0.003 0.009 0.006 5.71 mprotect : 8 0.006 0.039 0.016 32.16 munmap : 12 0.007 0.077 0.020 38.25 brk : 48 0.002 0.014 0.004 10.18 rt_sigaction : 18 0.002 0.002 0.002 2.11 rt_sigprocmask : 60 0.002 0.128 0.010 32.88 access : 2 0.006 0.006 0.006 0.00 pipe : 12 0.004 0.048 0.013 35.98 vfork : 34 0.448 0.980 0.692 3.04 execve : 20 0.000 0.387 0.046 56.66 wait4 : 34 0.017 9923.287 593.221 68.45 fcntl : 8 0.001 0.041 0.013 48.79 getdents : 48 0.002 0.079 0.013 19.62 getcwd : 2 0.005 0.005 0.005 0.00 chdir : 2 0.070 0.070 0.070 0.00 getrlimit : 2 0.045 0.045 0.045 0.00 arch_prctl : 2 0.002 0.002 0.002 0.00 setrlimit : 2 0.002 0.002 0.002 0.00 openat : 94 0.003 0.005 0.003 2.11 Signed-off-by: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung.kim@lge.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1381289214-24885-3-git-send-email-dsahern@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-08 21:26:53 -06:00
perf machine: Protect the machine->threads with a rwlock In addition to using refcounts for the struct thread lifetime management, we need to protect access to machine->threads from concurrent access. That happens in 'perf top', where a thread processes events, inserting and deleting entries from that rb_tree while another thread decays hist_entries, that end up dropping references and ultimately deleting threads from the rb_tree and releasing its resources when no further hist_entry (or other data structures, like in 'perf sched') references it. So the rule is the same for refcounts + protected trees in the kernel, get the tree lock, find object, bump the refcount, drop the tree lock, return, use object, drop the refcount if no more use of it is needed, keep it if storing it in some other data structure, drop when releasing that data structure. I.e. pair "t = machine__find(new)_thread()" with a "thread__put(t)", and "perf_event__preprocess_sample(&al)" with "addr_location__put(&al)". The addr_location__put() one is because as we return references to several data structures, we may end up adding more reference counting for the other data structures and then we'll drop it at addr_location__put() time. Acked-by: David Ahern <dsahern@gmail.com> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-bs9rt4n0jw3hi9f3zxyy3xln@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-06 20:43:22 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf machine: Protect the machine->threads with a rwlock In addition to using refcounts for the struct thread lifetime management, we need to protect access to machine->threads from concurrent access. That happens in 'perf top', where a thread processes events, inserting and deleting entries from that rb_tree while another thread decays hist_entries, that end up dropping references and ultimately deleting threads from the rb_tree and releasing its resources when no further hist_entry (or other data structures, like in 'perf sched') references it. So the rule is the same for refcounts + protected trees in the kernel, get the tree lock, find object, bump the refcount, drop the tree lock, return, use object, drop the refcount if no more use of it is needed, keep it if storing it in some other data structure, drop when releasing that data structure. I.e. pair "t = machine__find(new)_thread()" with a "thread__put(t)", and "perf_event__preprocess_sample(&al)" with "addr_location__put(&al)". The addr_location__put() one is because as we return references to several data structures, we may end up adding more reference counting for the other data structures and then we'll drop it at addr_location__put() time. Acked-by: David Ahern <dsahern@gmail.com> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-bs9rt4n0jw3hi9f3zxyy3xln@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-06 20:43:22 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Fix syscall enter formatting bug commit e596663ebb28a068f5cca57f83285b7b293a2c83 Author: Arnaldo Carvalho de Melo <acme@redhat.com> Date: Fri Feb 13 13:22:21 2015 -0300 perf trace: Handle multiple threads better wrt syscalls being intermixed Introduced a bug where it considered the number of bytes output directly to the output file when formatting the syscall entry buffer that is stored to be finally printed at syscall exit, ending up leaving garbage at the start of syscalls that appeared while another syscall was being processed, in another thread. Fix it. Example of garbage in the output before this patch: 4280.102 ( 0.000 ms): lsmd/763 ... [continued]: select()) = 0 Timeout 4280.107 (275.250 ms): tuned/852 select(tvp: 0x7f41f7ffde50 ) ... 4280.109 ( 0.002 ms): lsmd/763 Xl�� ) = -10 4639.197 ( 0.000 ms): systemd-journa/542 ... [continued]: epoll_wait()) = 1 4639.202 (359.088 ms): lsmd/763 select(n: 6, inp: 0x7ffff21daad0, tvp: 0x7ffff21daac0) ... 4639.207 ( 0.005 ms): systemd-journa/542 Hn�� ) = 106 4639.221 ( 0.002 ms): systemd-journa/542 uname(name: 0x7ffdbaed8e00) = 0 4639.271 ( 0.008 ms): systemd-journa/542 ftruncate(fd: 11</run/log/journal/60cd52417cf440a4a80107518bbd3c20/system.journal>, length: 50331648) = 0 Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-9ckfe8mvsedgkg6y80gz1ul8@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-03-25 18:01:15 -03:00
perf trace: Handle multiple threads better wrt syscalls being intermixed $ trace time taskset -c 0 usleep 1 0.845 ( 0.021 ms): time/16722 wait4(upid: 4294967295, stat_addr: 0x7fff17f443d4, ru: 0x7fff17f44438 ) ... 0.865 ( 0.008 ms): time/16723 execve(arg0: 140733595272004, arg1: 140733595272720, arg2: 140733595272768, arg3: 139755107218496, arg4: 7307199665339051828, arg5: 3) = -2 2.395 ( 1.523 ms): taskset/16723 execve(arg0: 140733595272013, arg1: 140733595272720, arg2: 140733595272768, arg3: 139755107218496, arg4: 7307199665339051828, arg5: 3) = 0 2.411 ( 0.002 ms): taskset/16723 brk( ) = 0x1915000 3.300 ( 0.058 ms): usleep/16723 nanosleep(rqtp: 0x7ffff4ada190 ) = 0 <SNIP> 3.305 ( 0.000 ms): usleep/16723 exit_group( 3.363 ( 2.539 ms): time/16722 ... [continued]: wait4()) = 16723 3.366 ( 0.001 ms): time/16722 rt_sigaction(sig: INT, act: 0x7fff17f44160, oact: 0x7fff17f44200, sigsetsize: 8) = 0 We we're not seeing this line: 0.845 ( 0.021 ms): time/16722 wait4(upid: 4294967295, stat_addr: 0x7fff17f443d4, ru: 0x7fff17f44438 ) ... just the one when it finishes: 3.363 ( 2.539 ms): time/16722 ... [continued]: wait4()) = 16723 Still some issues left till we move to ordered_samples when multiple CPUs/threads are involved... Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-zq9x30a1ky3djqewqn2v3ja3@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-13 13:22:21 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf machine: Protect the machine->threads with a rwlock In addition to using refcounts for the struct thread lifetime management, we need to protect access to machine->threads from concurrent access. That happens in 'perf top', where a thread processes events, inserting and deleting entries from that rb_tree while another thread decays hist_entries, that end up dropping references and ultimately deleting threads from the rb_tree and releasing its resources when no further hist_entry (or other data structures, like in 'perf sched') references it. So the rule is the same for refcounts + protected trees in the kernel, get the tree lock, find object, bump the refcount, drop the tree lock, return, use object, drop the refcount if no more use of it is needed, keep it if storing it in some other data structure, drop when releasing that data structure. I.e. pair "t = machine__find(new)_thread()" with a "thread__put(t)", and "perf_event__preprocess_sample(&al)" with "addr_location__put(&al)". The addr_location__put() one is because as we return references to several data structures, we may end up adding more reference counting for the other data structures and then we'll drop it at addr_location__put() time. Acked-by: David Ahern <dsahern@gmail.com> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-bs9rt4n0jw3hi9f3zxyy3xln@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-06 20:43:22 -03:00
perf machine: Protect the machine->threads with a rwlock In addition to using refcounts for the struct thread lifetime management, we need to protect access to machine->threads from concurrent access. That happens in 'perf top', where a thread processes events, inserting and deleting entries from that rb_tree while another thread decays hist_entries, that end up dropping references and ultimately deleting threads from the rb_tree and releasing its resources when no further hist_entry (or other data structures, like in 'perf sched') references it. So the rule is the same for refcounts + protected trees in the kernel, get the tree lock, find object, bump the refcount, drop the tree lock, return, use object, drop the refcount if no more use of it is needed, keep it if storing it in some other data structure, drop when releasing that data structure. I.e. pair "t = machine__find(new)_thread()" with a "thread__put(t)", and "perf_event__preprocess_sample(&al)" with "addr_location__put(&al)". The addr_location__put() one is because as we return references to several data structures, we may end up adding more reference counting for the other data structures and then we'll drop it at addr_location__put() time. Acked-by: David Ahern <dsahern@gmail.com> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-bs9rt4n0jw3hi9f3zxyy3xln@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-06 20:43:22 -03:00
perf trace: Add summary option to dump syscall statistics When enabled dumps a summary of all syscalls by task with the usual statistics -- min, max, average and relative stddev. For example, make - 26341 : 3344 [ 17.4% ] 0.000 ms read : 52 0.000 4.802 0.644 30.08 write : 20 0.004 0.036 0.010 21.72 open : 24 0.003 0.046 0.014 23.68 close : 64 0.002 0.055 0.008 22.53 stat : 2714 0.002 0.222 0.004 4.47 fstat : 18 0.001 0.041 0.006 46.26 mmap : 30 0.003 0.009 0.006 5.71 mprotect : 8 0.006 0.039 0.016 32.16 munmap : 12 0.007 0.077 0.020 38.25 brk : 48 0.002 0.014 0.004 10.18 rt_sigaction : 18 0.002 0.002 0.002 2.11 rt_sigprocmask : 60 0.002 0.128 0.010 32.88 access : 2 0.006 0.006 0.006 0.00 pipe : 12 0.004 0.048 0.013 35.98 vfork : 34 0.448 0.980 0.692 3.04 execve : 20 0.000 0.387 0.046 56.66 wait4 : 34 0.017 9923.287 593.221 68.45 fcntl : 8 0.001 0.041 0.013 48.79 getdents : 48 0.002 0.079 0.013 19.62 getcwd : 2 0.005 0.005 0.005 0.00 chdir : 2 0.070 0.070 0.070 0.00 getrlimit : 2 0.045 0.045 0.045 0.00 arch_prctl : 2 0.002 0.002 0.002 0.00 setrlimit : 2 0.002 0.002 0.002 0.00 openat : 94 0.003 0.005 0.003 2.11 Signed-off-by: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung.kim@lge.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1381289214-24885-3-git-send-email-dsahern@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-08 21:26:53 -06:00
perf machine: Protect the machine->threads with a rwlock In addition to using refcounts for the struct thread lifetime management, we need to protect access to machine->threads from concurrent access. That happens in 'perf top', where a thread processes events, inserting and deleting entries from that rb_tree while another thread decays hist_entries, that end up dropping references and ultimately deleting threads from the rb_tree and releasing its resources when no further hist_entry (or other data structures, like in 'perf sched') references it. So the rule is the same for refcounts + protected trees in the kernel, get the tree lock, find object, bump the refcount, drop the tree lock, return, use object, drop the refcount if no more use of it is needed, keep it if storing it in some other data structure, drop when releasing that data structure. I.e. pair "t = machine__find(new)_thread()" with a "thread__put(t)", and "perf_event__preprocess_sample(&al)" with "addr_location__put(&al)". The addr_location__put() one is because as we return references to several data structures, we may end up adding more reference counting for the other data structures and then we'll drop it at addr_location__put() time. Acked-by: David Ahern <dsahern@gmail.com> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-bs9rt4n0jw3hi9f3zxyy3xln@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-06 20:43:22 -03:00
perf trace: Add summary option to dump syscall statistics When enabled dumps a summary of all syscalls by task with the usual statistics -- min, max, average and relative stddev. For example, make - 26341 : 3344 [ 17.4% ] 0.000 ms read : 52 0.000 4.802 0.644 30.08 write : 20 0.004 0.036 0.010 21.72 open : 24 0.003 0.046 0.014 23.68 close : 64 0.002 0.055 0.008 22.53 stat : 2714 0.002 0.222 0.004 4.47 fstat : 18 0.001 0.041 0.006 46.26 mmap : 30 0.003 0.009 0.006 5.71 mprotect : 8 0.006 0.039 0.016 32.16 munmap : 12 0.007 0.077 0.020 38.25 brk : 48 0.002 0.014 0.004 10.18 rt_sigaction : 18 0.002 0.002 0.002 2.11 rt_sigprocmask : 60 0.002 0.128 0.010 32.88 access : 2 0.006 0.006 0.006 0.00 pipe : 12 0.004 0.048 0.013 35.98 vfork : 34 0.448 0.980 0.692 3.04 execve : 20 0.000 0.387 0.046 56.66 wait4 : 34 0.017 9923.287 593.221 68.45 fcntl : 8 0.001 0.041 0.013 48.79 getdents : 48 0.002 0.079 0.013 19.62 getcwd : 2 0.005 0.005 0.005 0.00 chdir : 2 0.070 0.070 0.070 0.00 getrlimit : 2 0.045 0.045 0.045 0.00 arch_prctl : 2 0.002 0.002 0.002 0.00 setrlimit : 2 0.002 0.002 0.002 0.00 openat : 94 0.003 0.005 0.003 2.11 Signed-off-by: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung.kim@lge.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1381289214-24885-3-git-send-email-dsahern@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-08 21:26:53 -06:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf machine: Protect the machine->threads with a rwlock In addition to using refcounts for the struct thread lifetime management, we need to protect access to machine->threads from concurrent access. That happens in 'perf top', where a thread processes events, inserting and deleting entries from that rb_tree while another thread decays hist_entries, that end up dropping references and ultimately deleting threads from the rb_tree and releasing its resources when no further hist_entry (or other data structures, like in 'perf sched') references it. So the rule is the same for refcounts + protected trees in the kernel, get the tree lock, find object, bump the refcount, drop the tree lock, return, use object, drop the refcount if no more use of it is needed, keep it if storing it in some other data structure, drop when releasing that data structure. I.e. pair "t = machine__find(new)_thread()" with a "thread__put(t)", and "perf_event__preprocess_sample(&al)" with "addr_location__put(&al)". The addr_location__put() one is because as we return references to several data structures, we may end up adding more reference counting for the other data structures and then we'll drop it at addr_location__put() time. Acked-by: David Ahern <dsahern@gmail.com> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-bs9rt4n0jw3hi9f3zxyy3xln@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-06 20:43:22 -03:00
perf trace: Deref sys_enter pointer args with contents from probe:vfs_getname To work like strace and dereference syscall pointer args we need to insert probes (or tracepoints) right after we copy those bytes from userspace. Since we're formatting the syscall args at raw_syscalls:sys_enter time, we need to have a formatter that just stores the position where, later, when we get the probe:vfs_getname, we can insert the pointer contents. Now, if a probe:vfs_getname with this format is in place: # perf probe -l probe:vfs_getname (on getname_flags:72@/home/git/linux/fs/namei.c with pathname) That was, in this case, put in place with: # perf probe 'vfs_getname=getname_flags:72 pathname=filename:string' Added new event: probe:vfs_getname (on getname_flags:72 with pathname=filename:string) You can now use it in all perf tools, such as: perf record -e probe:vfs_getname -aR sleep 1 # Then 'perf trace' will notice that and do the pointer -> contents expansion: # trace -e open touch /tmp/bla 0.165 (0.010 ms): touch/17752 open(filename: /etc/ld.so.cache, flags: CLOEXEC) = 3 0.195 (0.011 ms): touch/17752 open(filename: /lib64/libc.so.6, flags: CLOEXEC) = 3 0.512 (0.012 ms): touch/17752 open(filename: /usr/lib/locale/locale-archive, flags: CLOEXEC) = 3 0.582 (0.012 ms): touch/17752 open(filename: /tmp/bla, flags: CREAT|NOCTTY|NONBLOCK|WRONLY, mode: 438) = 3 # Roughly equivalent to strace's output: # strace -rT -e open touch /tmp/bla 0.000000 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 <0.000039> 0.000317 open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 <0.000102> 0.001461 open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3 <0.000072> 0.000405 open("/tmp/bla", O_WRONLY|O_CREAT|O_NOCTTY|O_NONBLOCK, 0666) = 3 <0.000055> 0.000641 +++ exited with 0 +++ # Now we need to either look for at all syscalls that are marked as pointers and have some well known names ("filename", "pathname", etc) and set the arg formatter to the one used for the "open" syscall in this patch. This implementation works for syscalls with just a string being copied from userspace, for matching syscalls with more than one string being copied via the same probe/trace point (vfs_getname) we need to extend the vfs_getname probe spec to include the pointer too, but there are some problems with that in 'perf probe' or the kernel kprobes code, need to investigate before considering supporting multiple strings per syscall. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Milian Wolff <mail@milianw.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-xvuwx6nuj8cf389kf9s2ue2s@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-08-04 22:30:09 -03:00
perf trace: Deref sys_enter pointer args with contents from probe:vfs_getname To work like strace and dereference syscall pointer args we need to insert probes (or tracepoints) right after we copy those bytes from userspace. Since we're formatting the syscall args at raw_syscalls:sys_enter time, we need to have a formatter that just stores the position where, later, when we get the probe:vfs_getname, we can insert the pointer contents. Now, if a probe:vfs_getname with this format is in place: # perf probe -l probe:vfs_getname (on getname_flags:72@/home/git/linux/fs/namei.c with pathname) That was, in this case, put in place with: # perf probe 'vfs_getname=getname_flags:72 pathname=filename:string' Added new event: probe:vfs_getname (on getname_flags:72 with pathname=filename:string) You can now use it in all perf tools, such as: perf record -e probe:vfs_getname -aR sleep 1 # Then 'perf trace' will notice that and do the pointer -> contents expansion: # trace -e open touch /tmp/bla 0.165 (0.010 ms): touch/17752 open(filename: /etc/ld.so.cache, flags: CLOEXEC) = 3 0.195 (0.011 ms): touch/17752 open(filename: /lib64/libc.so.6, flags: CLOEXEC) = 3 0.512 (0.012 ms): touch/17752 open(filename: /usr/lib/locale/locale-archive, flags: CLOEXEC) = 3 0.582 (0.012 ms): touch/17752 open(filename: /tmp/bla, flags: CREAT|NOCTTY|NONBLOCK|WRONLY, mode: 438) = 3 # Roughly equivalent to strace's output: # strace -rT -e open touch /tmp/bla 0.000000 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 <0.000039> 0.000317 open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 <0.000102> 0.001461 open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3 <0.000072> 0.000405 open("/tmp/bla", O_WRONLY|O_CREAT|O_NOCTTY|O_NONBLOCK, 0666) = 3 <0.000055> 0.000641 +++ exited with 0 +++ # Now we need to either look for at all syscalls that are marked as pointers and have some well known names ("filename", "pathname", etc) and set the arg formatter to the one used for the "open" syscall in this patch. This implementation works for syscalls with just a string being copied from userspace, for matching syscalls with more than one string being copied via the same probe/trace point (vfs_getname) we need to extend the vfs_getname probe spec to include the pointer too, but there are some problems with that in 'perf probe' or the kernel kprobes code, need to investigate before considering supporting multiple strings per syscall. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Milian Wolff <mail@milianw.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-xvuwx6nuj8cf389kf9s2ue2s@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-08-04 22:30:09 -03:00
perf trace: Deref sys_enter pointer args with contents from probe:vfs_getname To work like strace and dereference syscall pointer args we need to insert probes (or tracepoints) right after we copy those bytes from userspace. Since we're formatting the syscall args at raw_syscalls:sys_enter time, we need to have a formatter that just stores the position where, later, when we get the probe:vfs_getname, we can insert the pointer contents. Now, if a probe:vfs_getname with this format is in place: # perf probe -l probe:vfs_getname (on getname_flags:72@/home/git/linux/fs/namei.c with pathname) That was, in this case, put in place with: # perf probe 'vfs_getname=getname_flags:72 pathname=filename:string' Added new event: probe:vfs_getname (on getname_flags:72 with pathname=filename:string) You can now use it in all perf tools, such as: perf record -e probe:vfs_getname -aR sleep 1 # Then 'perf trace' will notice that and do the pointer -> contents expansion: # trace -e open touch /tmp/bla 0.165 (0.010 ms): touch/17752 open(filename: /etc/ld.so.cache, flags: CLOEXEC) = 3 0.195 (0.011 ms): touch/17752 open(filename: /lib64/libc.so.6, flags: CLOEXEC) = 3 0.512 (0.012 ms): touch/17752 open(filename: /usr/lib/locale/locale-archive, flags: CLOEXEC) = 3 0.582 (0.012 ms): touch/17752 open(filename: /tmp/bla, flags: CREAT|NOCTTY|NONBLOCK|WRONLY, mode: 438) = 3 # Roughly equivalent to strace's output: # strace -rT -e open touch /tmp/bla 0.000000 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 <0.000039> 0.000317 open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 <0.000102> 0.001461 open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3 <0.000072> 0.000405 open("/tmp/bla", O_WRONLY|O_CREAT|O_NOCTTY|O_NONBLOCK, 0666) = 3 <0.000055> 0.000641 +++ exited with 0 +++ # Now we need to either look for at all syscalls that are marked as pointers and have some well known names ("filename", "pathname", etc) and set the arg formatter to the one used for the "open" syscall in this patch. This implementation works for syscalls with just a string being copied from userspace, for matching syscalls with more than one string being copied via the same probe/trace point (vfs_getname) we need to extend the vfs_getname probe spec to include the pointer too, but there are some problems with that in 'perf probe' or the kernel kprobes code, need to investigate before considering supporting multiple strings per syscall. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Milian Wolff <mail@milianw.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-xvuwx6nuj8cf389kf9s2ue2s@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-08-04 22:30:09 -03:00
perf trace: Use sched:sched_stat_runtime to provide a thread summary [root@sandy ~]# perf trace --sched --duration 0.100 --pid `pidof firefox` <SNIP> 17079.847 ( 0.009 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.892 ( 0.010 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable 17079.921 ( 0.013 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.949 ( 0.009 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ firefox - 17643 : 18013 [ 72.2% ] 359.110 ms firefox - 17663 : 41 [ 0.2% ] 21.439 ms firefox - 17664 : 6840 [ 27.4% ] 133.642 ms firefox - 17667 : 46 [ 0.2% ] 0.682 ms [root@sandy ~]# This is equivalent to the 'perf trace summary' subcomand in the tmp.perf/trace2 branch. Another example, setting a huge duration filter to get just a system wide summary: [root@sandy ~]# perf trace --duration 10000.0 --sched ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ scsi_eh_1 - 258 : 15 [ 0.0% ] 0.133 ms kworker/0:1H - 322 : 13 [ 0.0% ] 0.032 ms jbd2/dm-0-8 - 384 : 4 [ 0.0% ] 0.115 ms flush-253:0 - 470 : 1 [ 0.0% ] 0.027 ms firefox - 950 : 4783 [ 0.1% ] 24.863 ms firefox - 992 : 1883 [ 0.1% ] 6.808 ms firefox - 995 : 35 [ 0.0% ] 0.111 ms ksoftirqd/6 - 4362 : 2 [ 0.0% ] 0.005 ms ksoftirqd/7 - 4365 : 1 [ 0.0% ] 0.007 ms Xorg - 4671 : 148 [ 0.0% ] 0.912 ms gnome-settings- - 4846 : 14 [ 0.0% ] 0.086 ms seahorse-daemon - 4847 : 14 [ 0.0% ] 0.092 ms gnome-panel - 4875 : 46 [ 0.0% ] 0.159 ms gnome-power-man - 4918 : 16 [ 0.0% ] 0.065 ms gvfs-afc-volume - 4992 : 77 [ 0.0% ] 0.136 ms gnome-screensav - 5114 : 24 [ 0.0% ] 0.128 ms xchat - 8082 : 466 [ 0.0% ] 2.019 ms synergyc - 8369 : 941 [ 0.0% ] 3.291 ms synergyc - 8371 : 85 [ 0.0% ] 1.817 ms jbd2/dm-4-8 - 9352 : 4 [ 0.0% ] 0.109 ms rpcbind - 9786 : 3 [ 0.0% ] 0.017 ms rtkit-daemon - 12802 : 10 [ 0.0% ] 0.038 ms rtkit-daemon - 12803 : 8 [ 0.0% ] 0.000 ms udisks-daemon - 13020 : 27 [ 0.0% ] 0.240 ms kworker/7:0 - 14651 : 669 [ 0.0% ] 2.616 ms kworker/5:1 - 16220 : 2 [ 0.0% ] 0.069 ms kworker/4:0 - 19776 : 13 [ 0.0% ] 0.176 ms openvpn - 20131 : 133 [ 0.0% ] 0.762 ms plugin-containe - 20508 : 60658 [ 1.7% ] 131.153 ms npviewer.bin - 20520 : 72208 [ 2.0% ] 138.945 ms npviewer.bin - 20542 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 20543 : 30 [ 0.0% ] 0.074 ms npviewer.bin - 20547 : 35 [ 0.0% ] 0.092 ms npviewer.bin - 20552 : 35 [ 0.0% ] 0.093 ms sshd - 20645 : 32 [ 0.0% ] 0.071 ms npviewer.bin - 21053 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 21054 : 35 [ 0.0% ] 0.097 ms kworker/0:2 - 21169 : 149 [ 0.0% ] 1.143 ms kworker/3:0 - 22171 : 113 [ 0.0% ] 96.892 ms flush-253:4 - 22410 : 1 [ 0.0% ] 0.028 ms kworker/6:0 - 24581 : 25 [ 0.0% ] 0.275 ms kworker/1:0 - 25572 : 4 [ 0.0% ] 0.103 ms kworker/2:1 - 26299 : 138 [ 0.0% ] 1.440 ms kworker/0:0 - 26325 : 1 [ 0.0% ] 0.003 ms perf - 26330 : 3506967 [ 96.1% ] 6648.310 ms [root@sandy ~]# Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-mzuli0srnxyi1o029py6537x@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-17 17:13:12 -03:00
perf trace: Use sched:sched_stat_runtime to provide a thread summary [root@sandy ~]# perf trace --sched --duration 0.100 --pid `pidof firefox` <SNIP> 17079.847 ( 0.009 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.892 ( 0.010 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable 17079.921 ( 0.013 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.949 ( 0.009 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ firefox - 17643 : 18013 [ 72.2% ] 359.110 ms firefox - 17663 : 41 [ 0.2% ] 21.439 ms firefox - 17664 : 6840 [ 27.4% ] 133.642 ms firefox - 17667 : 46 [ 0.2% ] 0.682 ms [root@sandy ~]# This is equivalent to the 'perf trace summary' subcomand in the tmp.perf/trace2 branch. Another example, setting a huge duration filter to get just a system wide summary: [root@sandy ~]# perf trace --duration 10000.0 --sched ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ scsi_eh_1 - 258 : 15 [ 0.0% ] 0.133 ms kworker/0:1H - 322 : 13 [ 0.0% ] 0.032 ms jbd2/dm-0-8 - 384 : 4 [ 0.0% ] 0.115 ms flush-253:0 - 470 : 1 [ 0.0% ] 0.027 ms firefox - 950 : 4783 [ 0.1% ] 24.863 ms firefox - 992 : 1883 [ 0.1% ] 6.808 ms firefox - 995 : 35 [ 0.0% ] 0.111 ms ksoftirqd/6 - 4362 : 2 [ 0.0% ] 0.005 ms ksoftirqd/7 - 4365 : 1 [ 0.0% ] 0.007 ms Xorg - 4671 : 148 [ 0.0% ] 0.912 ms gnome-settings- - 4846 : 14 [ 0.0% ] 0.086 ms seahorse-daemon - 4847 : 14 [ 0.0% ] 0.092 ms gnome-panel - 4875 : 46 [ 0.0% ] 0.159 ms gnome-power-man - 4918 : 16 [ 0.0% ] 0.065 ms gvfs-afc-volume - 4992 : 77 [ 0.0% ] 0.136 ms gnome-screensav - 5114 : 24 [ 0.0% ] 0.128 ms xchat - 8082 : 466 [ 0.0% ] 2.019 ms synergyc - 8369 : 941 [ 0.0% ] 3.291 ms synergyc - 8371 : 85 [ 0.0% ] 1.817 ms jbd2/dm-4-8 - 9352 : 4 [ 0.0% ] 0.109 ms rpcbind - 9786 : 3 [ 0.0% ] 0.017 ms rtkit-daemon - 12802 : 10 [ 0.0% ] 0.038 ms rtkit-daemon - 12803 : 8 [ 0.0% ] 0.000 ms udisks-daemon - 13020 : 27 [ 0.0% ] 0.240 ms kworker/7:0 - 14651 : 669 [ 0.0% ] 2.616 ms kworker/5:1 - 16220 : 2 [ 0.0% ] 0.069 ms kworker/4:0 - 19776 : 13 [ 0.0% ] 0.176 ms openvpn - 20131 : 133 [ 0.0% ] 0.762 ms plugin-containe - 20508 : 60658 [ 1.7% ] 131.153 ms npviewer.bin - 20520 : 72208 [ 2.0% ] 138.945 ms npviewer.bin - 20542 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 20543 : 30 [ 0.0% ] 0.074 ms npviewer.bin - 20547 : 35 [ 0.0% ] 0.092 ms npviewer.bin - 20552 : 35 [ 0.0% ] 0.093 ms sshd - 20645 : 32 [ 0.0% ] 0.071 ms npviewer.bin - 21053 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 21054 : 35 [ 0.0% ] 0.097 ms kworker/0:2 - 21169 : 149 [ 0.0% ] 1.143 ms kworker/3:0 - 22171 : 113 [ 0.0% ] 96.892 ms flush-253:4 - 22410 : 1 [ 0.0% ] 0.028 ms kworker/6:0 - 24581 : 25 [ 0.0% ] 0.275 ms kworker/1:0 - 25572 : 4 [ 0.0% ] 0.103 ms kworker/2:1 - 26299 : 138 [ 0.0% ] 1.440 ms kworker/0:0 - 26325 : 1 [ 0.0% ] 0.003 ms perf - 26330 : 3506967 [ 96.1% ] 6648.310 ms [root@sandy ~]# Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-mzuli0srnxyi1o029py6537x@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-17 17:13:12 -03:00
perf trace: Use sched:sched_stat_runtime to provide a thread summary [root@sandy ~]# perf trace --sched --duration 0.100 --pid `pidof firefox` <SNIP> 17079.847 ( 0.009 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.892 ( 0.010 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable 17079.921 ( 0.013 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.949 ( 0.009 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ firefox - 17643 : 18013 [ 72.2% ] 359.110 ms firefox - 17663 : 41 [ 0.2% ] 21.439 ms firefox - 17664 : 6840 [ 27.4% ] 133.642 ms firefox - 17667 : 46 [ 0.2% ] 0.682 ms [root@sandy ~]# This is equivalent to the 'perf trace summary' subcomand in the tmp.perf/trace2 branch. Another example, setting a huge duration filter to get just a system wide summary: [root@sandy ~]# perf trace --duration 10000.0 --sched ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ scsi_eh_1 - 258 : 15 [ 0.0% ] 0.133 ms kworker/0:1H - 322 : 13 [ 0.0% ] 0.032 ms jbd2/dm-0-8 - 384 : 4 [ 0.0% ] 0.115 ms flush-253:0 - 470 : 1 [ 0.0% ] 0.027 ms firefox - 950 : 4783 [ 0.1% ] 24.863 ms firefox - 992 : 1883 [ 0.1% ] 6.808 ms firefox - 995 : 35 [ 0.0% ] 0.111 ms ksoftirqd/6 - 4362 : 2 [ 0.0% ] 0.005 ms ksoftirqd/7 - 4365 : 1 [ 0.0% ] 0.007 ms Xorg - 4671 : 148 [ 0.0% ] 0.912 ms gnome-settings- - 4846 : 14 [ 0.0% ] 0.086 ms seahorse-daemon - 4847 : 14 [ 0.0% ] 0.092 ms gnome-panel - 4875 : 46 [ 0.0% ] 0.159 ms gnome-power-man - 4918 : 16 [ 0.0% ] 0.065 ms gvfs-afc-volume - 4992 : 77 [ 0.0% ] 0.136 ms gnome-screensav - 5114 : 24 [ 0.0% ] 0.128 ms xchat - 8082 : 466 [ 0.0% ] 2.019 ms synergyc - 8369 : 941 [ 0.0% ] 3.291 ms synergyc - 8371 : 85 [ 0.0% ] 1.817 ms jbd2/dm-4-8 - 9352 : 4 [ 0.0% ] 0.109 ms rpcbind - 9786 : 3 [ 0.0% ] 0.017 ms rtkit-daemon - 12802 : 10 [ 0.0% ] 0.038 ms rtkit-daemon - 12803 : 8 [ 0.0% ] 0.000 ms udisks-daemon - 13020 : 27 [ 0.0% ] 0.240 ms kworker/7:0 - 14651 : 669 [ 0.0% ] 2.616 ms kworker/5:1 - 16220 : 2 [ 0.0% ] 0.069 ms kworker/4:0 - 19776 : 13 [ 0.0% ] 0.176 ms openvpn - 20131 : 133 [ 0.0% ] 0.762 ms plugin-containe - 20508 : 60658 [ 1.7% ] 131.153 ms npviewer.bin - 20520 : 72208 [ 2.0% ] 138.945 ms npviewer.bin - 20542 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 20543 : 30 [ 0.0% ] 0.074 ms npviewer.bin - 20547 : 35 [ 0.0% ] 0.092 ms npviewer.bin - 20552 : 35 [ 0.0% ] 0.093 ms sshd - 20645 : 32 [ 0.0% ] 0.071 ms npviewer.bin - 21053 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 21054 : 35 [ 0.0% ] 0.097 ms kworker/0:2 - 21169 : 149 [ 0.0% ] 1.143 ms kworker/3:0 - 22171 : 113 [ 0.0% ] 96.892 ms flush-253:4 - 22410 : 1 [ 0.0% ] 0.028 ms kworker/6:0 - 24581 : 25 [ 0.0% ] 0.275 ms kworker/1:0 - 25572 : 4 [ 0.0% ] 0.103 ms kworker/2:1 - 26299 : 138 [ 0.0% ] 1.440 ms kworker/0:0 - 26325 : 1 [ 0.0% ] 0.003 ms perf - 26330 : 3506967 [ 96.1% ] 6648.310 ms [root@sandy ~]# Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-mzuli0srnxyi1o029py6537x@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-17 17:13:12 -03:00
perf machine: Protect the machine->threads with a rwlock In addition to using refcounts for the struct thread lifetime management, we need to protect access to machine->threads from concurrent access. That happens in 'perf top', where a thread processes events, inserting and deleting entries from that rb_tree while another thread decays hist_entries, that end up dropping references and ultimately deleting threads from the rb_tree and releasing its resources when no further hist_entry (or other data structures, like in 'perf sched') references it. So the rule is the same for refcounts + protected trees in the kernel, get the tree lock, find object, bump the refcount, drop the tree lock, return, use object, drop the refcount if no more use of it is needed, keep it if storing it in some other data structure, drop when releasing that data structure. I.e. pair "t = machine__find(new)_thread()" with a "thread__put(t)", and "perf_event__preprocess_sample(&al)" with "addr_location__put(&al)". The addr_location__put() one is because as we return references to several data structures, we may end up adding more reference counting for the other data structures and then we'll drop it at addr_location__put() time. Acked-by: David Ahern <dsahern@gmail.com> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-bs9rt4n0jw3hi9f3zxyy3xln@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-06 20:43:22 -03:00
perf trace: Use sched:sched_stat_runtime to provide a thread summary [root@sandy ~]# perf trace --sched --duration 0.100 --pid `pidof firefox` <SNIP> 17079.847 ( 0.009 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.892 ( 0.010 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable 17079.921 ( 0.013 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.949 ( 0.009 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ firefox - 17643 : 18013 [ 72.2% ] 359.110 ms firefox - 17663 : 41 [ 0.2% ] 21.439 ms firefox - 17664 : 6840 [ 27.4% ] 133.642 ms firefox - 17667 : 46 [ 0.2% ] 0.682 ms [root@sandy ~]# This is equivalent to the 'perf trace summary' subcomand in the tmp.perf/trace2 branch. Another example, setting a huge duration filter to get just a system wide summary: [root@sandy ~]# perf trace --duration 10000.0 --sched ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ scsi_eh_1 - 258 : 15 [ 0.0% ] 0.133 ms kworker/0:1H - 322 : 13 [ 0.0% ] 0.032 ms jbd2/dm-0-8 - 384 : 4 [ 0.0% ] 0.115 ms flush-253:0 - 470 : 1 [ 0.0% ] 0.027 ms firefox - 950 : 4783 [ 0.1% ] 24.863 ms firefox - 992 : 1883 [ 0.1% ] 6.808 ms firefox - 995 : 35 [ 0.0% ] 0.111 ms ksoftirqd/6 - 4362 : 2 [ 0.0% ] 0.005 ms ksoftirqd/7 - 4365 : 1 [ 0.0% ] 0.007 ms Xorg - 4671 : 148 [ 0.0% ] 0.912 ms gnome-settings- - 4846 : 14 [ 0.0% ] 0.086 ms seahorse-daemon - 4847 : 14 [ 0.0% ] 0.092 ms gnome-panel - 4875 : 46 [ 0.0% ] 0.159 ms gnome-power-man - 4918 : 16 [ 0.0% ] 0.065 ms gvfs-afc-volume - 4992 : 77 [ 0.0% ] 0.136 ms gnome-screensav - 5114 : 24 [ 0.0% ] 0.128 ms xchat - 8082 : 466 [ 0.0% ] 2.019 ms synergyc - 8369 : 941 [ 0.0% ] 3.291 ms synergyc - 8371 : 85 [ 0.0% ] 1.817 ms jbd2/dm-4-8 - 9352 : 4 [ 0.0% ] 0.109 ms rpcbind - 9786 : 3 [ 0.0% ] 0.017 ms rtkit-daemon - 12802 : 10 [ 0.0% ] 0.038 ms rtkit-daemon - 12803 : 8 [ 0.0% ] 0.000 ms udisks-daemon - 13020 : 27 [ 0.0% ] 0.240 ms kworker/7:0 - 14651 : 669 [ 0.0% ] 2.616 ms kworker/5:1 - 16220 : 2 [ 0.0% ] 0.069 ms kworker/4:0 - 19776 : 13 [ 0.0% ] 0.176 ms openvpn - 20131 : 133 [ 0.0% ] 0.762 ms plugin-containe - 20508 : 60658 [ 1.7% ] 131.153 ms npviewer.bin - 20520 : 72208 [ 2.0% ] 138.945 ms npviewer.bin - 20542 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 20543 : 30 [ 0.0% ] 0.074 ms npviewer.bin - 20547 : 35 [ 0.0% ] 0.092 ms npviewer.bin - 20552 : 35 [ 0.0% ] 0.093 ms sshd - 20645 : 32 [ 0.0% ] 0.071 ms npviewer.bin - 21053 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 21054 : 35 [ 0.0% ] 0.097 ms kworker/0:2 - 21169 : 149 [ 0.0% ] 1.143 ms kworker/3:0 - 22171 : 113 [ 0.0% ] 96.892 ms flush-253:4 - 22410 : 1 [ 0.0% ] 0.028 ms kworker/6:0 - 24581 : 25 [ 0.0% ] 0.275 ms kworker/1:0 - 25572 : 4 [ 0.0% ] 0.103 ms kworker/2:1 - 26299 : 138 [ 0.0% ] 1.440 ms kworker/0:0 - 26325 : 1 [ 0.0% ] 0.003 ms perf - 26330 : 3506967 [ 96.1% ] 6648.310 ms [root@sandy ~]# Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-mzuli0srnxyi1o029py6537x@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-17 17:13:12 -03:00
perf trace: Use sched:sched_stat_runtime to provide a thread summary [root@sandy ~]# perf trace --sched --duration 0.100 --pid `pidof firefox` <SNIP> 17079.847 ( 0.009 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.892 ( 0.010 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable 17079.921 ( 0.013 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.949 ( 0.009 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ firefox - 17643 : 18013 [ 72.2% ] 359.110 ms firefox - 17663 : 41 [ 0.2% ] 21.439 ms firefox - 17664 : 6840 [ 27.4% ] 133.642 ms firefox - 17667 : 46 [ 0.2% ] 0.682 ms [root@sandy ~]# This is equivalent to the 'perf trace summary' subcomand in the tmp.perf/trace2 branch. Another example, setting a huge duration filter to get just a system wide summary: [root@sandy ~]# perf trace --duration 10000.0 --sched ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ scsi_eh_1 - 258 : 15 [ 0.0% ] 0.133 ms kworker/0:1H - 322 : 13 [ 0.0% ] 0.032 ms jbd2/dm-0-8 - 384 : 4 [ 0.0% ] 0.115 ms flush-253:0 - 470 : 1 [ 0.0% ] 0.027 ms firefox - 950 : 4783 [ 0.1% ] 24.863 ms firefox - 992 : 1883 [ 0.1% ] 6.808 ms firefox - 995 : 35 [ 0.0% ] 0.111 ms ksoftirqd/6 - 4362 : 2 [ 0.0% ] 0.005 ms ksoftirqd/7 - 4365 : 1 [ 0.0% ] 0.007 ms Xorg - 4671 : 148 [ 0.0% ] 0.912 ms gnome-settings- - 4846 : 14 [ 0.0% ] 0.086 ms seahorse-daemon - 4847 : 14 [ 0.0% ] 0.092 ms gnome-panel - 4875 : 46 [ 0.0% ] 0.159 ms gnome-power-man - 4918 : 16 [ 0.0% ] 0.065 ms gvfs-afc-volume - 4992 : 77 [ 0.0% ] 0.136 ms gnome-screensav - 5114 : 24 [ 0.0% ] 0.128 ms xchat - 8082 : 466 [ 0.0% ] 2.019 ms synergyc - 8369 : 941 [ 0.0% ] 3.291 ms synergyc - 8371 : 85 [ 0.0% ] 1.817 ms jbd2/dm-4-8 - 9352 : 4 [ 0.0% ] 0.109 ms rpcbind - 9786 : 3 [ 0.0% ] 0.017 ms rtkit-daemon - 12802 : 10 [ 0.0% ] 0.038 ms rtkit-daemon - 12803 : 8 [ 0.0% ] 0.000 ms udisks-daemon - 13020 : 27 [ 0.0% ] 0.240 ms kworker/7:0 - 14651 : 669 [ 0.0% ] 2.616 ms kworker/5:1 - 16220 : 2 [ 0.0% ] 0.069 ms kworker/4:0 - 19776 : 13 [ 0.0% ] 0.176 ms openvpn - 20131 : 133 [ 0.0% ] 0.762 ms plugin-containe - 20508 : 60658 [ 1.7% ] 131.153 ms npviewer.bin - 20520 : 72208 [ 2.0% ] 138.945 ms npviewer.bin - 20542 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 20543 : 30 [ 0.0% ] 0.074 ms npviewer.bin - 20547 : 35 [ 0.0% ] 0.092 ms npviewer.bin - 20552 : 35 [ 0.0% ] 0.093 ms sshd - 20645 : 32 [ 0.0% ] 0.071 ms npviewer.bin - 21053 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 21054 : 35 [ 0.0% ] 0.097 ms kworker/0:2 - 21169 : 149 [ 0.0% ] 1.143 ms kworker/3:0 - 22171 : 113 [ 0.0% ] 96.892 ms flush-253:4 - 22410 : 1 [ 0.0% ] 0.028 ms kworker/6:0 - 24581 : 25 [ 0.0% ] 0.275 ms kworker/1:0 - 25572 : 4 [ 0.0% ] 0.103 ms kworker/2:1 - 26299 : 138 [ 0.0% ] 1.440 ms kworker/0:0 - 26325 : 1 [ 0.0% ] 0.003 ms perf - 26330 : 3506967 [ 96.1% ] 6648.310 ms [root@sandy ~]# Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-mzuli0srnxyi1o029py6537x@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-17 17:13:12 -03:00
perf machine: Protect the machine->threads with a rwlock In addition to using refcounts for the struct thread lifetime management, we need to protect access to machine->threads from concurrent access. That happens in 'perf top', where a thread processes events, inserting and deleting entries from that rb_tree while another thread decays hist_entries, that end up dropping references and ultimately deleting threads from the rb_tree and releasing its resources when no further hist_entry (or other data structures, like in 'perf sched') references it. So the rule is the same for refcounts + protected trees in the kernel, get the tree lock, find object, bump the refcount, drop the tree lock, return, use object, drop the refcount if no more use of it is needed, keep it if storing it in some other data structure, drop when releasing that data structure. I.e. pair "t = machine__find(new)_thread()" with a "thread__put(t)", and "perf_event__preprocess_sample(&al)" with "addr_location__put(&al)". The addr_location__put() one is because as we return references to several data structures, we may end up adding more reference counting for the other data structures and then we'll drop it at addr_location__put() time. Acked-by: David Ahern <dsahern@gmail.com> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-bs9rt4n0jw3hi9f3zxyy3xln@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-06 20:43:22 -03:00
perf trace: Use sched:sched_stat_runtime to provide a thread summary [root@sandy ~]# perf trace --sched --duration 0.100 --pid `pidof firefox` <SNIP> 17079.847 ( 0.009 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.892 ( 0.010 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable 17079.921 ( 0.013 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.949 ( 0.009 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ firefox - 17643 : 18013 [ 72.2% ] 359.110 ms firefox - 17663 : 41 [ 0.2% ] 21.439 ms firefox - 17664 : 6840 [ 27.4% ] 133.642 ms firefox - 17667 : 46 [ 0.2% ] 0.682 ms [root@sandy ~]# This is equivalent to the 'perf trace summary' subcomand in the tmp.perf/trace2 branch. Another example, setting a huge duration filter to get just a system wide summary: [root@sandy ~]# perf trace --duration 10000.0 --sched ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ scsi_eh_1 - 258 : 15 [ 0.0% ] 0.133 ms kworker/0:1H - 322 : 13 [ 0.0% ] 0.032 ms jbd2/dm-0-8 - 384 : 4 [ 0.0% ] 0.115 ms flush-253:0 - 470 : 1 [ 0.0% ] 0.027 ms firefox - 950 : 4783 [ 0.1% ] 24.863 ms firefox - 992 : 1883 [ 0.1% ] 6.808 ms firefox - 995 : 35 [ 0.0% ] 0.111 ms ksoftirqd/6 - 4362 : 2 [ 0.0% ] 0.005 ms ksoftirqd/7 - 4365 : 1 [ 0.0% ] 0.007 ms Xorg - 4671 : 148 [ 0.0% ] 0.912 ms gnome-settings- - 4846 : 14 [ 0.0% ] 0.086 ms seahorse-daemon - 4847 : 14 [ 0.0% ] 0.092 ms gnome-panel - 4875 : 46 [ 0.0% ] 0.159 ms gnome-power-man - 4918 : 16 [ 0.0% ] 0.065 ms gvfs-afc-volume - 4992 : 77 [ 0.0% ] 0.136 ms gnome-screensav - 5114 : 24 [ 0.0% ] 0.128 ms xchat - 8082 : 466 [ 0.0% ] 2.019 ms synergyc - 8369 : 941 [ 0.0% ] 3.291 ms synergyc - 8371 : 85 [ 0.0% ] 1.817 ms jbd2/dm-4-8 - 9352 : 4 [ 0.0% ] 0.109 ms rpcbind - 9786 : 3 [ 0.0% ] 0.017 ms rtkit-daemon - 12802 : 10 [ 0.0% ] 0.038 ms rtkit-daemon - 12803 : 8 [ 0.0% ] 0.000 ms udisks-daemon - 13020 : 27 [ 0.0% ] 0.240 ms kworker/7:0 - 14651 : 669 [ 0.0% ] 2.616 ms kworker/5:1 - 16220 : 2 [ 0.0% ] 0.069 ms kworker/4:0 - 19776 : 13 [ 0.0% ] 0.176 ms openvpn - 20131 : 133 [ 0.0% ] 0.762 ms plugin-containe - 20508 : 60658 [ 1.7% ] 131.153 ms npviewer.bin - 20520 : 72208 [ 2.0% ] 138.945 ms npviewer.bin - 20542 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 20543 : 30 [ 0.0% ] 0.074 ms npviewer.bin - 20547 : 35 [ 0.0% ] 0.092 ms npviewer.bin - 20552 : 35 [ 0.0% ] 0.093 ms sshd - 20645 : 32 [ 0.0% ] 0.071 ms npviewer.bin - 21053 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 21054 : 35 [ 0.0% ] 0.097 ms kworker/0:2 - 21169 : 149 [ 0.0% ] 1.143 ms kworker/3:0 - 22171 : 113 [ 0.0% ] 96.892 ms flush-253:4 - 22410 : 1 [ 0.0% ] 0.028 ms kworker/6:0 - 24581 : 25 [ 0.0% ] 0.275 ms kworker/1:0 - 25572 : 4 [ 0.0% ] 0.103 ms kworker/2:1 - 26299 : 138 [ 0.0% ] 1.440 ms kworker/0:0 - 26325 : 1 [ 0.0% ] 0.003 ms perf - 26330 : 3506967 [ 96.1% ] 6648.310 ms [root@sandy ~]# Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-mzuli0srnxyi1o029py6537x@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-17 17:13:12 -03:00
perf trace: Allow mixing with other events Basically adopting 'perf record' --event command line argument syntax: # trace -e \!mprotect,mmap,munmap,open,close,read,fstat,access,arch_prctl --event sched:*switch,sched:*exec,sched:*exit usleep 1 0.048 ( ): sched:sched_process_exec:filename=/bin/usleep pid=24732 old_pid=24732) 0.078 (0.002 ms): usleep/24732 brk( ) = 0x78f000 0.430 (0.002 ms): usleep/24732 brk( ) = 0x78f000 0.434 (0.003 ms): usleep/24732 brk(brk: 0x7b0000 ) = 0x7b0000 0.438 (0.001 ms): usleep/24732 brk( ) = 0x7b0000 0.460 (0.004 ms): usleep/24732 nanosleep(rqtp: 0x7ffff3696a40) ... 0.460 ( ): sched:sched_switch:prev_comm=usleep prev_pid=24732 prev_prio=120 prev_state=S ==> next_comm=swapper/1 next_pid=0 next_prio=120) 0.515 (0.058 ms): usleep/24732 ... [continued]: nanosleep()) = 0 0.520 (0.000 ms): usleep/24732 exit_group( 0.550 ( ): sched:sched_process_exit:comm=usleep pid=24732 prio=120) # Next steps, probably in this order: 1) Use ordered_events code, the logic in trace needs the events to be time ordered when needed, i.e. when multiple CPUs are involved. 2) Callchains! 3) Automatically account for interruptions when saying how long things took. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-gpst8mph575yb4wgf91qibyb@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-03 12:58:57 -03:00
perf trace: Allow mixing with other events Basically adopting 'perf record' --event command line argument syntax: # trace -e \!mprotect,mmap,munmap,open,close,read,fstat,access,arch_prctl --event sched:*switch,sched:*exec,sched:*exit usleep 1 0.048 ( ): sched:sched_process_exec:filename=/bin/usleep pid=24732 old_pid=24732) 0.078 (0.002 ms): usleep/24732 brk( ) = 0x78f000 0.430 (0.002 ms): usleep/24732 brk( ) = 0x78f000 0.434 (0.003 ms): usleep/24732 brk(brk: 0x7b0000 ) = 0x7b0000 0.438 (0.001 ms): usleep/24732 brk( ) = 0x7b0000 0.460 (0.004 ms): usleep/24732 nanosleep(rqtp: 0x7ffff3696a40) ... 0.460 ( ): sched:sched_switch:prev_comm=usleep prev_pid=24732 prev_prio=120 prev_state=S ==> next_comm=swapper/1 next_pid=0 next_prio=120) 0.515 (0.058 ms): usleep/24732 ... [continued]: nanosleep()) = 0 0.520 (0.000 ms): usleep/24732 exit_group( 0.550 ( ): sched:sched_process_exit:comm=usleep pid=24732 prio=120) # Next steps, probably in this order: 1) Use ordered_events code, the logic in trace needs the events to be time ordered when needed, i.e. when multiple CPUs are involved. 2) Callchains! 3) Automatically account for interruptions when saying how long things took. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-gpst8mph575yb4wgf91qibyb@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-03 12:58:57 -03:00
2014-07-08 15:39:21 -03:00
2014-07-08 15:39:21 -03:00
2014-07-08 15:39:21 -03:00
perf machine: Protect the machine->threads with a rwlock In addition to using refcounts for the struct thread lifetime management, we need to protect access to machine->threads from concurrent access. That happens in 'perf top', where a thread processes events, inserting and deleting entries from that rb_tree while another thread decays hist_entries, that end up dropping references and ultimately deleting threads from the rb_tree and releasing its resources when no further hist_entry (or other data structures, like in 'perf sched') references it. So the rule is the same for refcounts + protected trees in the kernel, get the tree lock, find object, bump the refcount, drop the tree lock, return, use object, drop the refcount if no more use of it is needed, keep it if storing it in some other data structure, drop when releasing that data structure. I.e. pair "t = machine__find(new)_thread()" with a "thread__put(t)", and "perf_event__preprocess_sample(&al)" with "addr_location__put(&al)". The addr_location__put() one is because as we return references to several data structures, we may end up adding more reference counting for the other data structures and then we'll drop it at addr_location__put() time. Acked-by: David Ahern <dsahern@gmail.com> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-bs9rt4n0jw3hi9f3zxyy3xln@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-06 20:43:22 -03:00
perf machine: Protect the machine->threads with a rwlock In addition to using refcounts for the struct thread lifetime management, we need to protect access to machine->threads from concurrent access. That happens in 'perf top', where a thread processes events, inserting and deleting entries from that rb_tree while another thread decays hist_entries, that end up dropping references and ultimately deleting threads from the rb_tree and releasing its resources when no further hist_entry (or other data structures, like in 'perf sched') references it. So the rule is the same for refcounts + protected trees in the kernel, get the tree lock, find object, bump the refcount, drop the tree lock, return, use object, drop the refcount if no more use of it is needed, keep it if storing it in some other data structure, drop when releasing that data structure. I.e. pair "t = machine__find(new)_thread()" with a "thread__put(t)", and "perf_event__preprocess_sample(&al)" with "addr_location__put(&al)". The addr_location__put() one is because as we return references to several data structures, we may end up adding more reference counting for the other data structures and then we'll drop it at addr_location__put() time. Acked-by: David Ahern <dsahern@gmail.com> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-bs9rt4n0jw3hi9f3zxyy3xln@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-06 20:43:22 -03:00
perf machine: Protect the machine->threads with a rwlock In addition to using refcounts for the struct thread lifetime management, we need to protect access to machine->threads from concurrent access. That happens in 'perf top', where a thread processes events, inserting and deleting entries from that rb_tree while another thread decays hist_entries, that end up dropping references and ultimately deleting threads from the rb_tree and releasing its resources when no further hist_entry (or other data structures, like in 'perf sched') references it. So the rule is the same for refcounts + protected trees in the kernel, get the tree lock, find object, bump the refcount, drop the tree lock, return, use object, drop the refcount if no more use of it is needed, keep it if storing it in some other data structure, drop when releasing that data structure. I.e. pair "t = machine__find(new)_thread()" with a "thread__put(t)", and "perf_event__preprocess_sample(&al)" with "addr_location__put(&al)". The addr_location__put() one is because as we return references to several data structures, we may end up adding more reference counting for the other data structures and then we'll drop it at addr_location__put() time. Acked-by: David Ahern <dsahern@gmail.com> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-bs9rt4n0jw3hi9f3zxyy3xln@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-06 20:43:22 -03:00
perf machine: Protect the machine->threads with a rwlock In addition to using refcounts for the struct thread lifetime management, we need to protect access to machine->threads from concurrent access. That happens in 'perf top', where a thread processes events, inserting and deleting entries from that rb_tree while another thread decays hist_entries, that end up dropping references and ultimately deleting threads from the rb_tree and releasing its resources when no further hist_entry (or other data structures, like in 'perf sched') references it. So the rule is the same for refcounts + protected trees in the kernel, get the tree lock, find object, bump the refcount, drop the tree lock, return, use object, drop the refcount if no more use of it is needed, keep it if storing it in some other data structure, drop when releasing that data structure. I.e. pair "t = machine__find(new)_thread()" with a "thread__put(t)", and "perf_event__preprocess_sample(&al)" with "addr_location__put(&al)". The addr_location__put() one is because as we return references to several data structures, we may end up adding more reference counting for the other data structures and then we'll drop it at addr_location__put() time. Acked-by: David Ahern <dsahern@gmail.com> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-bs9rt4n0jw3hi9f3zxyy3xln@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-06 20:43:22 -03:00
perf trace: Add pagefaults record and replay support Previous commit added live pagefault trace support, this one adds record and replay support. Example: [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace -F all record -a sleep 10 [ perf record: Woken up 0 times to write data ] [ perf record: Captured and wrote 1029.722 MB perf.data (~44989242 samples) ] [root@zoo /]# ls -la perf.data -rw-------. 1 root root 1083921722 Jun 26 17:44 perf.data [root@zoo /]# perf evlist raw_syscalls:sys_enter raw_syscalls:sys_exit major-faults minor-faults [root@zoo /]# trace -i perf.data | grep -v trace\/ | tail -15 156.137 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.139 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) 156.140 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0xc4] => 0x0 (?.) 156.144 ( 0.000 ms): perl/18476 minfault [_int_free+0xda] => 0x0 (?.) 156.151 ( 0.000 ms): perl/18476 minfault [_int_free+0x1df] => 0x0 (?.) 156.158 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.161 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.168 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.172 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.173 ( 0.000 ms): perl/18476 minfault [_int_free+0xda] => 0x0 (?.) 156.183 ( 0.000 ms): perl/18476 minfault [Perl_hfree_next_entry+0xb4] => 0x0 (?.) 156.197 ( 0.000 ms): perl/18476 minfault [_int_free+0x1df] => 0x0 (?.) 156.216 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) 156.221 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) [root@zoo /]# Signed-off-by: Stanislav Fomichev <stfomichev@yandex-team.ru> Cc: David Ahern <dsahern@gmail.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1403799268-1367-4-git-send-email-stfomichev@yandex-team.ru Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-06-26 20:14:26 +04:00
perf trace: Add pagefaults record and replay support Previous commit added live pagefault trace support, this one adds record and replay support. Example: [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace -F all record -a sleep 10 [ perf record: Woken up 0 times to write data ] [ perf record: Captured and wrote 1029.722 MB perf.data (~44989242 samples) ] [root@zoo /]# ls -la perf.data -rw-------. 1 root root 1083921722 Jun 26 17:44 perf.data [root@zoo /]# perf evlist raw_syscalls:sys_enter raw_syscalls:sys_exit major-faults minor-faults [root@zoo /]# trace -i perf.data | grep -v trace\/ | tail -15 156.137 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.139 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) 156.140 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0xc4] => 0x0 (?.) 156.144 ( 0.000 ms): perl/18476 minfault [_int_free+0xda] => 0x0 (?.) 156.151 ( 0.000 ms): perl/18476 minfault [_int_free+0x1df] => 0x0 (?.) 156.158 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.161 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.168 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.172 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.173 ( 0.000 ms): perl/18476 minfault [_int_free+0xda] => 0x0 (?.) 156.183 ( 0.000 ms): perl/18476 minfault [Perl_hfree_next_entry+0xb4] => 0x0 (?.) 156.197 ( 0.000 ms): perl/18476 minfault [_int_free+0x1df] => 0x0 (?.) 156.216 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) 156.221 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) [root@zoo /]# Signed-off-by: Stanislav Fomichev <stfomichev@yandex-team.ru> Cc: David Ahern <dsahern@gmail.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1403799268-1367-4-git-send-email-stfomichev@yandex-team.ru Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-06-26 20:14:26 +04:00
perf trace: Add pagefaults record and replay support Previous commit added live pagefault trace support, this one adds record and replay support. Example: [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace -F all record -a sleep 10 [ perf record: Woken up 0 times to write data ] [ perf record: Captured and wrote 1029.722 MB perf.data (~44989242 samples) ] [root@zoo /]# ls -la perf.data -rw-------. 1 root root 1083921722 Jun 26 17:44 perf.data [root@zoo /]# perf evlist raw_syscalls:sys_enter raw_syscalls:sys_exit major-faults minor-faults [root@zoo /]# trace -i perf.data | grep -v trace\/ | tail -15 156.137 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.139 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) 156.140 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0xc4] => 0x0 (?.) 156.144 ( 0.000 ms): perl/18476 minfault [_int_free+0xda] => 0x0 (?.) 156.151 ( 0.000 ms): perl/18476 minfault [_int_free+0x1df] => 0x0 (?.) 156.158 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.161 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.168 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.172 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.173 ( 0.000 ms): perl/18476 minfault [_int_free+0xda] => 0x0 (?.) 156.183 ( 0.000 ms): perl/18476 minfault [Perl_hfree_next_entry+0xb4] => 0x0 (?.) 156.197 ( 0.000 ms): perl/18476 minfault [_int_free+0x1df] => 0x0 (?.) 156.216 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) 156.221 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) [root@zoo /]# Signed-off-by: Stanislav Fomichev <stfomichev@yandex-team.ru> Cc: David Ahern <dsahern@gmail.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1403799268-1367-4-git-send-email-stfomichev@yandex-team.ru Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-06-26 20:14:26 +04:00
perf trace: Add pagefaults record and replay support Previous commit added live pagefault trace support, this one adds record and replay support. Example: [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace -F all record -a sleep 10 [ perf record: Woken up 0 times to write data ] [ perf record: Captured and wrote 1029.722 MB perf.data (~44989242 samples) ] [root@zoo /]# ls -la perf.data -rw-------. 1 root root 1083921722 Jun 26 17:44 perf.data [root@zoo /]# perf evlist raw_syscalls:sys_enter raw_syscalls:sys_exit major-faults minor-faults [root@zoo /]# trace -i perf.data | grep -v trace\/ | tail -15 156.137 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.139 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) 156.140 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0xc4] => 0x0 (?.) 156.144 ( 0.000 ms): perl/18476 minfault [_int_free+0xda] => 0x0 (?.) 156.151 ( 0.000 ms): perl/18476 minfault [_int_free+0x1df] => 0x0 (?.) 156.158 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.161 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.168 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.172 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.173 ( 0.000 ms): perl/18476 minfault [_int_free+0xda] => 0x0 (?.) 156.183 ( 0.000 ms): perl/18476 minfault [Perl_hfree_next_entry+0xb4] => 0x0 (?.) 156.197 ( 0.000 ms): perl/18476 minfault [_int_free+0x1df] => 0x0 (?.) 156.216 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) 156.221 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) [root@zoo /]# Signed-off-by: Stanislav Fomichev <stfomichev@yandex-team.ru> Cc: David Ahern <dsahern@gmail.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1403799268-1367-4-git-send-email-stfomichev@yandex-team.ru Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-06-26 20:14:26 +04:00
perf trace: Add pagefaults record and replay support Previous commit added live pagefault trace support, this one adds record and replay support. Example: [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace -F all record -a sleep 10 [ perf record: Woken up 0 times to write data ] [ perf record: Captured and wrote 1029.722 MB perf.data (~44989242 samples) ] [root@zoo /]# ls -la perf.data -rw-------. 1 root root 1083921722 Jun 26 17:44 perf.data [root@zoo /]# perf evlist raw_syscalls:sys_enter raw_syscalls:sys_exit major-faults minor-faults [root@zoo /]# trace -i perf.data | grep -v trace\/ | tail -15 156.137 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.139 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) 156.140 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0xc4] => 0x0 (?.) 156.144 ( 0.000 ms): perl/18476 minfault [_int_free+0xda] => 0x0 (?.) 156.151 ( 0.000 ms): perl/18476 minfault [_int_free+0x1df] => 0x0 (?.) 156.158 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.161 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.168 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.172 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.173 ( 0.000 ms): perl/18476 minfault [_int_free+0xda] => 0x0 (?.) 156.183 ( 0.000 ms): perl/18476 minfault [Perl_hfree_next_entry+0xb4] => 0x0 (?.) 156.197 ( 0.000 ms): perl/18476 minfault [_int_free+0x1df] => 0x0 (?.) 156.216 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) 156.221 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) [root@zoo /]# Signed-off-by: Stanislav Fomichev <stfomichev@yandex-team.ru> Cc: David Ahern <dsahern@gmail.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1403799268-1367-4-git-send-email-stfomichev@yandex-team.ru Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-06-26 20:14:26 +04:00
perf trace: Add possibility to switch off syscall events Currently, we may either trace syscalls or syscalls+pagefaults. We'd like to be able to trace *only* pagefaults and this commit implements this feature. Example: [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace --no-syscalls -F -p `pidof xchat` 0.000 ( 0.000 ms): xchat/4574 majfault [g_unichar_get_script+0x11] => /usr/lib64/libglib-2.0.so.0.3800.2@0xc403b (x.) 0.202 ( 0.000 ms): xchat/4574 majfault [_cairo_hash_table_lookup+0x53] => 0x2280ff0 (?.) 20.854 ( 0.000 ms): xchat/4574 majfault [gdk_cairo_set_source_pixbuf+0x110] => /usr/bin/xchat@0x6da1f (x.) 1022.000 ( 0.000 ms): xchat/4574 majfault [__memcpy_sse2_unaligned+0x29] => 0x7ff5a8ca0400 (?.) ^C[root@zoo /]# Below we can see malloc calls, 'trace' reading symbol tables in libraries to resolve symbols, etc. [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace --no-syscalls -F all --cpu 1 sleep 10 0.000 ( 0.000 ms): chrome/26589 minfault [0x1b53129] => /tmp/perf-26589.map@0x33cbcbf7f000 (x.) 96.477 ( 0.000 ms): libvirtd/947 minfault [copy_user_enhanced_fast_string+0x5] => 0x7f7685bba000 (?k) 113.164 ( 0.000 ms): Xorg/1063 minfault [0x786da] => 0x7fce52882a3c (?.) 7162.801 ( 0.000 ms): chrome/3747 minfault [0x8e1a89] => 0xfcaefed0008 (?.) <SNIP> 7773.138 ( 0.000 ms): chrome/3886 minfault [0x8e1a89] => 0xfcb0ce28008 (?.) 7992.022 ( 0.000 ms): chrome/26574 minfault [0x1b5a708] => 0x3de7b5fc5000 (?.) 8108.949 ( 0.000 ms): qemu-system-x8/4537 majfault [_int_malloc+0xee] => 0x7faffc466d60 (?.) 8108.975 ( 0.000 ms): qemu-system-x8/4537 minfault [_int_malloc+0x102] => 0x7faffc466d60 (?.) <SNIP> 8148.174 ( 0.000 ms): qemu-system-x8/4537 minfault [_int_malloc+0x102] => 0x7faffc4eb500 (?.) 8270.855 ( 0.000 ms): chrome/26245 minfault [do_bo_emit_reloc+0xdb] => 0x45d092bc004 (?.) 8270.869 ( 0.000 ms): chrome/26245 minfault [do_bo_emit_reloc+0x108] => 0x45d09150000 (?.) no symbols found in /usr/lib64/libspice-server.so.1.9.0, maybe install a debug package? 8273.831 ( 0.000 ms): trace/20198 majfault [__memcmp_sse4_1+0xbc6] => /usr/lib64/libspice-server.so.1.9.0@0xdf000 (d.) <SNIP> 8275.121 ( 0.000 ms): trace/20198 minfault [dso__load+0x38] => 0x14fe756 (?.) no symbols found in /usr/lib64/libelf-0.158.so, maybe install a debug package? 8275.142 ( 0.000 ms): trace/20198 minfault [__memcmp_sse4_1+0xbc6] => /usr/lib64/libelf-0.158.so@0x0 (d.) <SNIP> [root@zoo /]# Signed-off-by: Stanislav Fomichev <stfomichev@yandex-team.ru> Cc: David Ahern <dsahern@gmail.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1403799268-1367-6-git-send-email-stfomichev@yandex-team.ru Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-06-26 20:14:28 +04:00
perf trace: Add possibility to switch off syscall events Currently, we may either trace syscalls or syscalls+pagefaults. We'd like to be able to trace *only* pagefaults and this commit implements this feature. Example: [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace --no-syscalls -F -p `pidof xchat` 0.000 ( 0.000 ms): xchat/4574 majfault [g_unichar_get_script+0x11] => /usr/lib64/libglib-2.0.so.0.3800.2@0xc403b (x.) 0.202 ( 0.000 ms): xchat/4574 majfault [_cairo_hash_table_lookup+0x53] => 0x2280ff0 (?.) 20.854 ( 0.000 ms): xchat/4574 majfault [gdk_cairo_set_source_pixbuf+0x110] => /usr/bin/xchat@0x6da1f (x.) 1022.000 ( 0.000 ms): xchat/4574 majfault [__memcpy_sse2_unaligned+0x29] => 0x7ff5a8ca0400 (?.) ^C[root@zoo /]# Below we can see malloc calls, 'trace' reading symbol tables in libraries to resolve symbols, etc. [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace --no-syscalls -F all --cpu 1 sleep 10 0.000 ( 0.000 ms): chrome/26589 minfault [0x1b53129] => /tmp/perf-26589.map@0x33cbcbf7f000 (x.) 96.477 ( 0.000 ms): libvirtd/947 minfault [copy_user_enhanced_fast_string+0x5] => 0x7f7685bba000 (?k) 113.164 ( 0.000 ms): Xorg/1063 minfault [0x786da] => 0x7fce52882a3c (?.) 7162.801 ( 0.000 ms): chrome/3747 minfault [0x8e1a89] => 0xfcaefed0008 (?.) <SNIP> 7773.138 ( 0.000 ms): chrome/3886 minfault [0x8e1a89] => 0xfcb0ce28008 (?.) 7992.022 ( 0.000 ms): chrome/26574 minfault [0x1b5a708] => 0x3de7b5fc5000 (?.) 8108.949 ( 0.000 ms): qemu-system-x8/4537 majfault [_int_malloc+0xee] => 0x7faffc466d60 (?.) 8108.975 ( 0.000 ms): qemu-system-x8/4537 minfault [_int_malloc+0x102] => 0x7faffc466d60 (?.) <SNIP> 8148.174 ( 0.000 ms): qemu-system-x8/4537 minfault [_int_malloc+0x102] => 0x7faffc4eb500 (?.) 8270.855 ( 0.000 ms): chrome/26245 minfault [do_bo_emit_reloc+0xdb] => 0x45d092bc004 (?.) 8270.869 ( 0.000 ms): chrome/26245 minfault [do_bo_emit_reloc+0x108] => 0x45d09150000 (?.) no symbols found in /usr/lib64/libspice-server.so.1.9.0, maybe install a debug package? 8273.831 ( 0.000 ms): trace/20198 majfault [__memcmp_sse4_1+0xbc6] => /usr/lib64/libspice-server.so.1.9.0@0xdf000 (d.) <SNIP> 8275.121 ( 0.000 ms): trace/20198 minfault [dso__load+0x38] => 0x14fe756 (?.) no symbols found in /usr/lib64/libelf-0.158.so, maybe install a debug package? 8275.142 ( 0.000 ms): trace/20198 minfault [__memcmp_sse4_1+0xbc6] => /usr/lib64/libelf-0.158.so@0x0 (d.) <SNIP> [root@zoo /]# Signed-off-by: Stanislav Fomichev <stfomichev@yandex-team.ru> Cc: David Ahern <dsahern@gmail.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1403799268-1367-6-git-send-email-stfomichev@yandex-team.ru Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-06-26 20:14:28 +04:00
perf trace: Add pagefaults record and replay support Previous commit added live pagefault trace support, this one adds record and replay support. Example: [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace -F all record -a sleep 10 [ perf record: Woken up 0 times to write data ] [ perf record: Captured and wrote 1029.722 MB perf.data (~44989242 samples) ] [root@zoo /]# ls -la perf.data -rw-------. 1 root root 1083921722 Jun 26 17:44 perf.data [root@zoo /]# perf evlist raw_syscalls:sys_enter raw_syscalls:sys_exit major-faults minor-faults [root@zoo /]# trace -i perf.data | grep -v trace\/ | tail -15 156.137 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.139 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) 156.140 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0xc4] => 0x0 (?.) 156.144 ( 0.000 ms): perl/18476 minfault [_int_free+0xda] => 0x0 (?.) 156.151 ( 0.000 ms): perl/18476 minfault [_int_free+0x1df] => 0x0 (?.) 156.158 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.161 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.168 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.172 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.173 ( 0.000 ms): perl/18476 minfault [_int_free+0xda] => 0x0 (?.) 156.183 ( 0.000 ms): perl/18476 minfault [Perl_hfree_next_entry+0xb4] => 0x0 (?.) 156.197 ( 0.000 ms): perl/18476 minfault [_int_free+0x1df] => 0x0 (?.) 156.216 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) 156.221 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) [root@zoo /]# Signed-off-by: Stanislav Fomichev <stfomichev@yandex-team.ru> Cc: David Ahern <dsahern@gmail.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1403799268-1367-4-git-send-email-stfomichev@yandex-team.ru Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-06-26 20:14:26 +04:00
perf trace: Add pagefaults record and replay support Previous commit added live pagefault trace support, this one adds record and replay support. Example: [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace -F all record -a sleep 10 [ perf record: Woken up 0 times to write data ] [ perf record: Captured and wrote 1029.722 MB perf.data (~44989242 samples) ] [root@zoo /]# ls -la perf.data -rw-------. 1 root root 1083921722 Jun 26 17:44 perf.data [root@zoo /]# perf evlist raw_syscalls:sys_enter raw_syscalls:sys_exit major-faults minor-faults [root@zoo /]# trace -i perf.data | grep -v trace\/ | tail -15 156.137 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.139 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) 156.140 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0xc4] => 0x0 (?.) 156.144 ( 0.000 ms): perl/18476 minfault [_int_free+0xda] => 0x0 (?.) 156.151 ( 0.000 ms): perl/18476 minfault [_int_free+0x1df] => 0x0 (?.) 156.158 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.161 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.168 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.172 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.173 ( 0.000 ms): perl/18476 minfault [_int_free+0xda] => 0x0 (?.) 156.183 ( 0.000 ms): perl/18476 minfault [Perl_hfree_next_entry+0xb4] => 0x0 (?.) 156.197 ( 0.000 ms): perl/18476 minfault [_int_free+0x1df] => 0x0 (?.) 156.216 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) 156.221 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) [root@zoo /]# Signed-off-by: Stanislav Fomichev <stfomichev@yandex-team.ru> Cc: David Ahern <dsahern@gmail.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1403799268-1367-4-git-send-email-stfomichev@yandex-team.ru Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-06-26 20:14:26 +04:00
perf trace: Add summary option to dump syscall statistics When enabled dumps a summary of all syscalls by task with the usual statistics -- min, max, average and relative stddev. For example, make - 26341 : 3344 [ 17.4% ] 0.000 ms read : 52 0.000 4.802 0.644 30.08 write : 20 0.004 0.036 0.010 21.72 open : 24 0.003 0.046 0.014 23.68 close : 64 0.002 0.055 0.008 22.53 stat : 2714 0.002 0.222 0.004 4.47 fstat : 18 0.001 0.041 0.006 46.26 mmap : 30 0.003 0.009 0.006 5.71 mprotect : 8 0.006 0.039 0.016 32.16 munmap : 12 0.007 0.077 0.020 38.25 brk : 48 0.002 0.014 0.004 10.18 rt_sigaction : 18 0.002 0.002 0.002 2.11 rt_sigprocmask : 60 0.002 0.128 0.010 32.88 access : 2 0.006 0.006 0.006 0.00 pipe : 12 0.004 0.048 0.013 35.98 vfork : 34 0.448 0.980 0.692 3.04 execve : 20 0.000 0.387 0.046 56.66 wait4 : 34 0.017 9923.287 593.221 68.45 fcntl : 8 0.001 0.041 0.013 48.79 getdents : 48 0.002 0.079 0.013 19.62 getcwd : 2 0.005 0.005 0.005 0.00 chdir : 2 0.070 0.070 0.070 0.00 getrlimit : 2 0.045 0.045 0.045 0.00 arch_prctl : 2 0.002 0.002 0.002 0.00 setrlimit : 2 0.002 0.002 0.002 0.00 openat : 94 0.003 0.005 0.003 2.11 Signed-off-by: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung.kim@lge.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1381289214-24885-3-git-send-email-dsahern@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-08 21:26:53 -06:00
perf trace: Use event filters for the event qualifier list We use raw_syscalls:sys_{enter,exit} events to show the syscalls, but were using a rather lazy/inneficient way to implement our 'strace -e' equivalent: filter out after reading the events in the ring buffer. Deflect more work to the kernel by appending a filter expression for that, that, together with the pid list, that is always present, if only to filter the tracer itself, reduces pressure on the ring buffer and otherwise use infrastructure already in place in the kernel to do early filtering. If we use it with -v we can see the filter passed to the kernel, for instance, for this contrieved case: # trace -v -e \!open,close,write,poll,recvfrom,select,recvmsg,writev,sendmsg,read,futex,epoll_wait,ioctl,eventfd --filter-pids 2189,2566,1398,2692,4475,4532 <SNIP> (common_pid != 2514 && common_pid != 1398 && common_pid != 2189 && common_pid != 2566 && common_pid != 2692 && common_pid != 4475 && common_pid != 4532) && (id != 3 && id != 232 && id != 284 && id != 202 && id != 16 && id != 2 && id != 7 && id != 0 && id != 45 && id != 47 && id != 23 && id != 46 && id != 1 && id != 20) 0.011 (0.011 ms): caribou/2295 eventfd2(flags: CLOEXEC|NONBLOCK) = 18 16.946 (0.019 ms): caribou/2295 eventfd2(flags: CLOEXEC|NONBLOCK) = 18 38.598 (0.167 ms): chronyd/794 socket(family: INET, type: DGRAM ) = 4 38.603 (0.002 ms): chronyd/794 fcntl(fd: 4<socket:[239307]>, cmd: GETFD) = 0 38.605 (0.001 ms): chronyd/794 fcntl(fd: 4<socket:[239307]>, cmd: SETFD, arg: 1) = 0 ^C # Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ti2tg18atproqpguc2moinp6@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-07-04 12:44:59 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Allow mixing with other events Basically adopting 'perf record' --event command line argument syntax: # trace -e \!mprotect,mmap,munmap,open,close,read,fstat,access,arch_prctl --event sched:*switch,sched:*exec,sched:*exit usleep 1 0.048 ( ): sched:sched_process_exec:filename=/bin/usleep pid=24732 old_pid=24732) 0.078 (0.002 ms): usleep/24732 brk( ) = 0x78f000 0.430 (0.002 ms): usleep/24732 brk( ) = 0x78f000 0.434 (0.003 ms): usleep/24732 brk(brk: 0x7b0000 ) = 0x7b0000 0.438 (0.001 ms): usleep/24732 brk( ) = 0x7b0000 0.460 (0.004 ms): usleep/24732 nanosleep(rqtp: 0x7ffff3696a40) ... 0.460 ( ): sched:sched_switch:prev_comm=usleep prev_pid=24732 prev_prio=120 prev_state=S ==> next_comm=swapper/1 next_pid=0 next_prio=120) 0.515 (0.058 ms): usleep/24732 ... [continued]: nanosleep()) = 0 0.520 (0.000 ms): usleep/24732 exit_group( 0.550 ( ): sched:sched_process_exit:comm=usleep pid=24732 prio=120) # Next steps, probably in this order: 1) Use ordered_events code, the logic in trace needs the events to be time ordered when needed, i.e. when multiple CPUs are involved. 2) Callchains! 3) Automatically account for interruptions when saying how long things took. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-gpst8mph575yb4wgf91qibyb@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-03 12:58:57 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Add possibility to switch off syscall events Currently, we may either trace syscalls or syscalls+pagefaults. We'd like to be able to trace *only* pagefaults and this commit implements this feature. Example: [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace --no-syscalls -F -p `pidof xchat` 0.000 ( 0.000 ms): xchat/4574 majfault [g_unichar_get_script+0x11] => /usr/lib64/libglib-2.0.so.0.3800.2@0xc403b (x.) 0.202 ( 0.000 ms): xchat/4574 majfault [_cairo_hash_table_lookup+0x53] => 0x2280ff0 (?.) 20.854 ( 0.000 ms): xchat/4574 majfault [gdk_cairo_set_source_pixbuf+0x110] => /usr/bin/xchat@0x6da1f (x.) 1022.000 ( 0.000 ms): xchat/4574 majfault [__memcpy_sse2_unaligned+0x29] => 0x7ff5a8ca0400 (?.) ^C[root@zoo /]# Below we can see malloc calls, 'trace' reading symbol tables in libraries to resolve symbols, etc. [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace --no-syscalls -F all --cpu 1 sleep 10 0.000 ( 0.000 ms): chrome/26589 minfault [0x1b53129] => /tmp/perf-26589.map@0x33cbcbf7f000 (x.) 96.477 ( 0.000 ms): libvirtd/947 minfault [copy_user_enhanced_fast_string+0x5] => 0x7f7685bba000 (?k) 113.164 ( 0.000 ms): Xorg/1063 minfault [0x786da] => 0x7fce52882a3c (?.) 7162.801 ( 0.000 ms): chrome/3747 minfault [0x8e1a89] => 0xfcaefed0008 (?.) <SNIP> 7773.138 ( 0.000 ms): chrome/3886 minfault [0x8e1a89] => 0xfcb0ce28008 (?.) 7992.022 ( 0.000 ms): chrome/26574 minfault [0x1b5a708] => 0x3de7b5fc5000 (?.) 8108.949 ( 0.000 ms): qemu-system-x8/4537 majfault [_int_malloc+0xee] => 0x7faffc466d60 (?.) 8108.975 ( 0.000 ms): qemu-system-x8/4537 minfault [_int_malloc+0x102] => 0x7faffc466d60 (?.) <SNIP> 8148.174 ( 0.000 ms): qemu-system-x8/4537 minfault [_int_malloc+0x102] => 0x7faffc4eb500 (?.) 8270.855 ( 0.000 ms): chrome/26245 minfault [do_bo_emit_reloc+0xdb] => 0x45d092bc004 (?.) 8270.869 ( 0.000 ms): chrome/26245 minfault [do_bo_emit_reloc+0x108] => 0x45d09150000 (?.) no symbols found in /usr/lib64/libspice-server.so.1.9.0, maybe install a debug package? 8273.831 ( 0.000 ms): trace/20198 majfault [__memcmp_sse4_1+0xbc6] => /usr/lib64/libspice-server.so.1.9.0@0xdf000 (d.) <SNIP> 8275.121 ( 0.000 ms): trace/20198 minfault [dso__load+0x38] => 0x14fe756 (?.) no symbols found in /usr/lib64/libelf-0.158.so, maybe install a debug package? 8275.142 ( 0.000 ms): trace/20198 minfault [__memcmp_sse4_1+0xbc6] => /usr/lib64/libelf-0.158.so@0x0 (d.) <SNIP> [root@zoo /]# Signed-off-by: Stanislav Fomichev <stfomichev@yandex-team.ru> Cc: David Ahern <dsahern@gmail.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1403799268-1367-6-git-send-email-stfomichev@yandex-team.ru Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-06-26 20:14:28 +04:00
perf trace: Use sched:sched_stat_runtime to provide a thread summary [root@sandy ~]# perf trace --sched --duration 0.100 --pid `pidof firefox` <SNIP> 17079.847 ( 0.009 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.892 ( 0.010 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable 17079.921 ( 0.013 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.949 ( 0.009 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ firefox - 17643 : 18013 [ 72.2% ] 359.110 ms firefox - 17663 : 41 [ 0.2% ] 21.439 ms firefox - 17664 : 6840 [ 27.4% ] 133.642 ms firefox - 17667 : 46 [ 0.2% ] 0.682 ms [root@sandy ~]# This is equivalent to the 'perf trace summary' subcomand in the tmp.perf/trace2 branch. Another example, setting a huge duration filter to get just a system wide summary: [root@sandy ~]# perf trace --duration 10000.0 --sched ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ scsi_eh_1 - 258 : 15 [ 0.0% ] 0.133 ms kworker/0:1H - 322 : 13 [ 0.0% ] 0.032 ms jbd2/dm-0-8 - 384 : 4 [ 0.0% ] 0.115 ms flush-253:0 - 470 : 1 [ 0.0% ] 0.027 ms firefox - 950 : 4783 [ 0.1% ] 24.863 ms firefox - 992 : 1883 [ 0.1% ] 6.808 ms firefox - 995 : 35 [ 0.0% ] 0.111 ms ksoftirqd/6 - 4362 : 2 [ 0.0% ] 0.005 ms ksoftirqd/7 - 4365 : 1 [ 0.0% ] 0.007 ms Xorg - 4671 : 148 [ 0.0% ] 0.912 ms gnome-settings- - 4846 : 14 [ 0.0% ] 0.086 ms seahorse-daemon - 4847 : 14 [ 0.0% ] 0.092 ms gnome-panel - 4875 : 46 [ 0.0% ] 0.159 ms gnome-power-man - 4918 : 16 [ 0.0% ] 0.065 ms gvfs-afc-volume - 4992 : 77 [ 0.0% ] 0.136 ms gnome-screensav - 5114 : 24 [ 0.0% ] 0.128 ms xchat - 8082 : 466 [ 0.0% ] 2.019 ms synergyc - 8369 : 941 [ 0.0% ] 3.291 ms synergyc - 8371 : 85 [ 0.0% ] 1.817 ms jbd2/dm-4-8 - 9352 : 4 [ 0.0% ] 0.109 ms rpcbind - 9786 : 3 [ 0.0% ] 0.017 ms rtkit-daemon - 12802 : 10 [ 0.0% ] 0.038 ms rtkit-daemon - 12803 : 8 [ 0.0% ] 0.000 ms udisks-daemon - 13020 : 27 [ 0.0% ] 0.240 ms kworker/7:0 - 14651 : 669 [ 0.0% ] 2.616 ms kworker/5:1 - 16220 : 2 [ 0.0% ] 0.069 ms kworker/4:0 - 19776 : 13 [ 0.0% ] 0.176 ms openvpn - 20131 : 133 [ 0.0% ] 0.762 ms plugin-containe - 20508 : 60658 [ 1.7% ] 131.153 ms npviewer.bin - 20520 : 72208 [ 2.0% ] 138.945 ms npviewer.bin - 20542 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 20543 : 30 [ 0.0% ] 0.074 ms npviewer.bin - 20547 : 35 [ 0.0% ] 0.092 ms npviewer.bin - 20552 : 35 [ 0.0% ] 0.093 ms sshd - 20645 : 32 [ 0.0% ] 0.071 ms npviewer.bin - 21053 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 21054 : 35 [ 0.0% ] 0.097 ms kworker/0:2 - 21169 : 149 [ 0.0% ] 1.143 ms kworker/3:0 - 22171 : 113 [ 0.0% ] 96.892 ms flush-253:4 - 22410 : 1 [ 0.0% ] 0.028 ms kworker/6:0 - 24581 : 25 [ 0.0% ] 0.275 ms kworker/1:0 - 25572 : 4 [ 0.0% ] 0.103 ms kworker/2:1 - 26299 : 138 [ 0.0% ] 1.440 ms kworker/0:0 - 26325 : 1 [ 0.0% ] 0.003 ms perf - 26330 : 3506967 [ 96.1% ] 6648.310 ms [root@sandy ~]# Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-mzuli0srnxyi1o029py6537x@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-17 17:13:12 -03:00
perf trace: Use sched:sched_stat_runtime to provide a thread summary [root@sandy ~]# perf trace --sched --duration 0.100 --pid `pidof firefox` <SNIP> 17079.847 ( 0.009 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.892 ( 0.010 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable 17079.921 ( 0.013 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.949 ( 0.009 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ firefox - 17643 : 18013 [ 72.2% ] 359.110 ms firefox - 17663 : 41 [ 0.2% ] 21.439 ms firefox - 17664 : 6840 [ 27.4% ] 133.642 ms firefox - 17667 : 46 [ 0.2% ] 0.682 ms [root@sandy ~]# This is equivalent to the 'perf trace summary' subcomand in the tmp.perf/trace2 branch. Another example, setting a huge duration filter to get just a system wide summary: [root@sandy ~]# perf trace --duration 10000.0 --sched ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ scsi_eh_1 - 258 : 15 [ 0.0% ] 0.133 ms kworker/0:1H - 322 : 13 [ 0.0% ] 0.032 ms jbd2/dm-0-8 - 384 : 4 [ 0.0% ] 0.115 ms flush-253:0 - 470 : 1 [ 0.0% ] 0.027 ms firefox - 950 : 4783 [ 0.1% ] 24.863 ms firefox - 992 : 1883 [ 0.1% ] 6.808 ms firefox - 995 : 35 [ 0.0% ] 0.111 ms ksoftirqd/6 - 4362 : 2 [ 0.0% ] 0.005 ms ksoftirqd/7 - 4365 : 1 [ 0.0% ] 0.007 ms Xorg - 4671 : 148 [ 0.0% ] 0.912 ms gnome-settings- - 4846 : 14 [ 0.0% ] 0.086 ms seahorse-daemon - 4847 : 14 [ 0.0% ] 0.092 ms gnome-panel - 4875 : 46 [ 0.0% ] 0.159 ms gnome-power-man - 4918 : 16 [ 0.0% ] 0.065 ms gvfs-afc-volume - 4992 : 77 [ 0.0% ] 0.136 ms gnome-screensav - 5114 : 24 [ 0.0% ] 0.128 ms xchat - 8082 : 466 [ 0.0% ] 2.019 ms synergyc - 8369 : 941 [ 0.0% ] 3.291 ms synergyc - 8371 : 85 [ 0.0% ] 1.817 ms jbd2/dm-4-8 - 9352 : 4 [ 0.0% ] 0.109 ms rpcbind - 9786 : 3 [ 0.0% ] 0.017 ms rtkit-daemon - 12802 : 10 [ 0.0% ] 0.038 ms rtkit-daemon - 12803 : 8 [ 0.0% ] 0.000 ms udisks-daemon - 13020 : 27 [ 0.0% ] 0.240 ms kworker/7:0 - 14651 : 669 [ 0.0% ] 2.616 ms kworker/5:1 - 16220 : 2 [ 0.0% ] 0.069 ms kworker/4:0 - 19776 : 13 [ 0.0% ] 0.176 ms openvpn - 20131 : 133 [ 0.0% ] 0.762 ms plugin-containe - 20508 : 60658 [ 1.7% ] 131.153 ms npviewer.bin - 20520 : 72208 [ 2.0% ] 138.945 ms npviewer.bin - 20542 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 20543 : 30 [ 0.0% ] 0.074 ms npviewer.bin - 20547 : 35 [ 0.0% ] 0.092 ms npviewer.bin - 20552 : 35 [ 0.0% ] 0.093 ms sshd - 20645 : 32 [ 0.0% ] 0.071 ms npviewer.bin - 21053 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 21054 : 35 [ 0.0% ] 0.097 ms kworker/0:2 - 21169 : 149 [ 0.0% ] 1.143 ms kworker/3:0 - 22171 : 113 [ 0.0% ] 96.892 ms flush-253:4 - 22410 : 1 [ 0.0% ] 0.028 ms kworker/6:0 - 24581 : 25 [ 0.0% ] 0.275 ms kworker/1:0 - 25572 : 4 [ 0.0% ] 0.103 ms kworker/2:1 - 26299 : 138 [ 0.0% ] 1.440 ms kworker/0:0 - 26325 : 1 [ 0.0% ] 0.003 ms perf - 26330 : 3506967 [ 96.1% ] 6648.310 ms [root@sandy ~]# Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-mzuli0srnxyi1o029py6537x@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-17 17:13:12 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: Support interrupted syscalls Using the same strategies as in the tmp.perf/trace2, i.e. the 'trace' tool implemented by tglx, just updated to the current codebase. Example: [root@sandy linux]# perf trace usleep 1 | tail 2.003: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128396288 2.017: mmap(addr: 0, len: 4096, prot: 3, flags: 34, fd: 4294967295, off: 0 ) = -2128400384 2.029: arch_prctl(option: 4098, arg2: 140146949441280, arg3: 140146949435392, arg4: 34, arg5: 4294967295) = 0 2.084: mprotect(start: 208741634048, len: 16384, prot: 1 ) = 0 2.098: mprotect(start: 208735956992, len: 4096, prot: 1 ) = 0 2.122: munmap(addr: 140146949447680, len: 91882 ) = 0 2.359: brk(brk: 0 ) = 28987392 2.371: brk(brk: 29122560 ) = 29122560 2.490: nanosleep(rqtp: 140735694241504, rmtp: 0 ) = 0 2.507: exit_group(error_code: 0 [root@sandy linux]# For now the timestamp and duration are always on, will be selectable. Also if multiple threads are being monitored, its tid will appear. The ret output continues to be interpreted a la strace. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ly9ulroru4my5isn0xe9gr0m@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-06 18:43:19 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Improve messages related to /proc/sys/kernel/perf_event_paranoid kernel/events/core.c has: /* * perf event paranoia level: * -1 - not paranoid at all * 0 - disallow raw tracepoint access for unpriv * 1 - disallow cpu events for unpriv * 2 - disallow kernel profiling for unpriv */ int sysctl_perf_event_paranoid __read_mostly = 1; So, with the default being 1, a non-root user can trace his stuff: [acme@zoo ~]$ cat /proc/sys/kernel/perf_event_paranoid 1 [acme@zoo ~]$ yes > /dev/null & [1] 15338 [acme@zoo ~]$ trace -p 15338 | head -5 0.005 ( 0.005 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 0.045 ( 0.001 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 0.085 ( 0.001 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 0.125 ( 0.001 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 0.165 ( 0.001 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 [acme@zoo ~]$ [acme@zoo ~]$ trace --duration 1 sleep 1 1002.148 (1001.218 ms): nanosleep(rqtp: 0x7fff46c79250 ) = 0 [acme@zoo ~]$ [acme@zoo ~]$ trace -- usleep 1 | tail -5 0.905 ( 0.002 ms): brk( ) = 0x1c82000 0.910 ( 0.003 ms): brk(brk: 0x1ca3000 ) = 0x1ca3000 0.913 ( 0.001 ms): brk( ) = 0x1ca3000 0.990 ( 0.059 ms): nanosleep(rqtp: 0x7fffe31a3280 ) = 0 0.995 ( 0.000 ms): exit_group( [acme@zoo ~]$ But can't do system wide tracing: [acme@zoo ~]$ trace Error: Operation not permitted. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 1. [acme@zoo ~]$ [acme@zoo ~]$ trace --cpu 0 Error: Operation not permitted. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 1. [acme@zoo ~]$ If the paranoid level is >= 2, i.e. turn this perf stuff off for !root users: [acme@zoo ~]$ sudo sh -c 'echo 2 > /proc/sys/kernel/perf_event_paranoid' [acme@zoo ~]$ cat /proc/sys/kernel/perf_event_paranoid 2 [acme@zoo ~]$ [acme@zoo ~]$ trace usleep 1 Error: Permission denied. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For your workloads it needs to be <= 1 Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 2. [acme@zoo ~]$ [acme@zoo ~]$ trace Error: Permission denied. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For your workloads it needs to be <= 1 Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 2. [acme@zoo ~]$ [acme@zoo ~]$ trace --cpu 1 Error: Permission denied. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For your workloads it needs to be <= 1 Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 2. [acme@zoo ~]$ If the user manages to get what he/she wants, convincing root not to be paranoid at all... [root@zoo ~]# echo -1 > /proc/sys/kernel/perf_event_paranoid [root@zoo ~]# cat /proc/sys/kernel/perf_event_paranoid -1 [root@zoo ~]# [acme@zoo ~]$ ps -eo user,pid,comm | grep Xorg root 729 Xorg [acme@zoo ~]$ [acme@zoo ~]$ trace -a --duration 0.001 -e \!select,ioctl,writev | grep Xorg | head -5 23.143 ( 0.003 ms): Xorg/729 setitimer(which: REAL, value: 0x7fffaadf16e0 ) = 0 23.152 ( 0.004 ms): Xorg/729 read(fd: 31, buf: 0x2544af0, count: 4096 ) = 8 23.161 ( 0.002 ms): Xorg/729 read(fd: 31, buf: 0x2544af0, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable 23.175 ( 0.002 ms): Xorg/729 setitimer(which: REAL, value: 0x7fffaadf16e0 ) = 0 23.235 ( 0.002 ms): Xorg/729 setitimer(which: REAL, value: 0x7fffaadf16e0 ) = 0 [acme@zoo ~]$ Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-di28olfwd28rvkox7v3hqhu1@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-17 17:38:29 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Introduce --filter-pids When tracing in X we get event loops due to the tracing activity, i.e. updates to a gnome-terminal that generate syscalls for X.org, etc. To get a more useful view of what is happening, syscall wise, system wide, we need to filter those, like in: # ps ax|egrep '981|2296|1519' | grep -v egrep 981 tty1 Ss+ 5:40 /usr/bin/Xorg :0 -background none ... 1519 ? Sl 2:22 /usr/bin/gnome-shell 2296 ? Sl 4:16 /usr/libexec/gnome-terminal-server # # trace -e write --filter-pids 981,2296,1519 0.385 ( 0.021 ms): goa-daemon/2061 write(fd: 1</dev/null>, buf: 0x7fbeb017b000, count: 136) = 136 0.922 ( 0.014 ms): goa-daemon/2061 write(fd: 1</dev/null>, buf: 0x7fbeb017b000, count: 140) = 140 5006.525 ( 0.029 ms): goa-daemon/2061 write(fd: 1</dev/null>, buf: 0x7fbeb017b000, count: 136) = 136 5007.235 ( 0.023 ms): goa-daemon/2061 write(fd: 1</dev/null>, buf: 0x7fbeb017b000, count: 140) = 140 5177.646 ( 0.018 ms): rtkit-daemon/782 write(fd: 5<anon_inode:[eventfd]>, buf: 0x7f7eea70be88, count: 8) = 8 8314.497 ( 0.004 ms): gsd-locate-poi/2084 write(fd: 5<anon_inode:[eventfd]>, buf: 0x7fffe96af7b0, count: 8) = 8 8314.518 ( 0.002 ms): gsd-locate-poi/2084 write(fd: 5<anon_inode:[eventfd]>, buf: 0x7fffe96af0e0, count: 8) = 8 ^C# When this option is used the tracer pid is also filtered. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-f5qmiyy7c0uxdm21ncatpeek@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-21 11:36:52 -08:00
perf trace: Introduce --filter-pids When tracing in X we get event loops due to the tracing activity, i.e. updates to a gnome-terminal that generate syscalls for X.org, etc. To get a more useful view of what is happening, syscall wise, system wide, we need to filter those, like in: # ps ax|egrep '981|2296|1519' | grep -v egrep 981 tty1 Ss+ 5:40 /usr/bin/Xorg :0 -background none ... 1519 ? Sl 2:22 /usr/bin/gnome-shell 2296 ? Sl 4:16 /usr/libexec/gnome-terminal-server # # trace -e write --filter-pids 981,2296,1519 0.385 ( 0.021 ms): goa-daemon/2061 write(fd: 1</dev/null>, buf: 0x7fbeb017b000, count: 136) = 136 0.922 ( 0.014 ms): goa-daemon/2061 write(fd: 1</dev/null>, buf: 0x7fbeb017b000, count: 140) = 140 5006.525 ( 0.029 ms): goa-daemon/2061 write(fd: 1</dev/null>, buf: 0x7fbeb017b000, count: 136) = 136 5007.235 ( 0.023 ms): goa-daemon/2061 write(fd: 1</dev/null>, buf: 0x7fbeb017b000, count: 140) = 140 5177.646 ( 0.018 ms): rtkit-daemon/782 write(fd: 5<anon_inode:[eventfd]>, buf: 0x7f7eea70be88, count: 8) = 8 8314.497 ( 0.004 ms): gsd-locate-poi/2084 write(fd: 5<anon_inode:[eventfd]>, buf: 0x7fffe96af7b0, count: 8) = 8 8314.518 ( 0.002 ms): gsd-locate-poi/2084 write(fd: 5<anon_inode:[eventfd]>, buf: 0x7fffe96af0e0, count: 8) = 8 ^C# When this option is used the tracer pid is also filtered. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-f5qmiyy7c0uxdm21ncatpeek@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-21 11:36:52 -08:00
perf trace: Use event filters for the event qualifier list We use raw_syscalls:sys_{enter,exit} events to show the syscalls, but were using a rather lazy/inneficient way to implement our 'strace -e' equivalent: filter out after reading the events in the ring buffer. Deflect more work to the kernel by appending a filter expression for that, that, together with the pid list, that is always present, if only to filter the tracer itself, reduces pressure on the ring buffer and otherwise use infrastructure already in place in the kernel to do early filtering. If we use it with -v we can see the filter passed to the kernel, for instance, for this contrieved case: # trace -v -e \!open,close,write,poll,recvfrom,select,recvmsg,writev,sendmsg,read,futex,epoll_wait,ioctl,eventfd --filter-pids 2189,2566,1398,2692,4475,4532 <SNIP> (common_pid != 2514 && common_pid != 1398 && common_pid != 2189 && common_pid != 2566 && common_pid != 2692 && common_pid != 4475 && common_pid != 4532) && (id != 3 && id != 232 && id != 284 && id != 202 && id != 16 && id != 2 && id != 7 && id != 0 && id != 45 && id != 47 && id != 23 && id != 46 && id != 1 && id != 20) 0.011 (0.011 ms): caribou/2295 eventfd2(flags: CLOEXEC|NONBLOCK) = 18 16.946 (0.019 ms): caribou/2295 eventfd2(flags: CLOEXEC|NONBLOCK) = 18 38.598 (0.167 ms): chronyd/794 socket(family: INET, type: DGRAM ) = 4 38.603 (0.002 ms): chronyd/794 fcntl(fd: 4<socket:[239307]>, cmd: GETFD) = 0 38.605 (0.001 ms): chronyd/794 fcntl(fd: 4<socket:[239307]>, cmd: SETFD, arg: 1) = 0 ^C # Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ti2tg18atproqpguc2moinp6@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-07-04 12:44:59 -03:00
perf trace: Use event filters for the event qualifier list We use raw_syscalls:sys_{enter,exit} events to show the syscalls, but were using a rather lazy/inneficient way to implement our 'strace -e' equivalent: filter out after reading the events in the ring buffer. Deflect more work to the kernel by appending a filter expression for that, that, together with the pid list, that is always present, if only to filter the tracer itself, reduces pressure on the ring buffer and otherwise use infrastructure already in place in the kernel to do early filtering. If we use it with -v we can see the filter passed to the kernel, for instance, for this contrieved case: # trace -v -e \!open,close,write,poll,recvfrom,select,recvmsg,writev,sendmsg,read,futex,epoll_wait,ioctl,eventfd --filter-pids 2189,2566,1398,2692,4475,4532 <SNIP> (common_pid != 2514 && common_pid != 1398 && common_pid != 2189 && common_pid != 2566 && common_pid != 2692 && common_pid != 4475 && common_pid != 4532) && (id != 3 && id != 232 && id != 284 && id != 202 && id != 16 && id != 2 && id != 7 && id != 0 && id != 45 && id != 47 && id != 23 && id != 46 && id != 1 && id != 20) 0.011 (0.011 ms): caribou/2295 eventfd2(flags: CLOEXEC|NONBLOCK) = 18 16.946 (0.019 ms): caribou/2295 eventfd2(flags: CLOEXEC|NONBLOCK) = 18 38.598 (0.167 ms): chronyd/794 socket(family: INET, type: DGRAM ) = 4 38.603 (0.002 ms): chronyd/794 fcntl(fd: 4<socket:[239307]>, cmd: GETFD) = 0 38.605 (0.001 ms): chronyd/794 fcntl(fd: 4<socket:[239307]>, cmd: SETFD, arg: 1) = 0 ^C # Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ti2tg18atproqpguc2moinp6@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-07-04 12:44:59 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Print thread info when following children The default for 'trace workload' is to set perf_event_attr.inherit to 1, i.e. to make it equivalent to 'strace -f workload', so we were ending with syscalls for multiple processes mixed up, fix it: Before: [root@ssdandy ~]# trace -e brk time usleep 1 0.071 ( 0.002 ms): brk( ) = 0x100e000 0.802 ( 0.001 ms): brk( ) = 0x1d99000 1.132 ( 0.003 ms): brk( ) = 0x1d99000 1.136 ( 0.003 ms): brk(brk: 0x1dba000) = 0x1dba000 1.140 ( 0.001 ms): brk( ) = 0x1dba000 0.00user 0.00system 0:00.00elapsed 63%CPU (0avgtext+0avgdata 528maxresident)k 0inputs+0outputs (0major+181minor)pagefaults 0swaps [root@ssdandy ~]# After: [root@ssdandy ~]# trace -f -e brk time usleep 1 0.072 ( 0.002 ms): time/26308 brk( ) = 0x1e6e000 0.860 ( 0.001 ms): usleep/26309 brk( ) = 0xb91000 1.193 ( 0.003 ms): usleep/26309 brk( ) = 0xb91000 1.197 ( 0.003 ms): usleep/26309 brk(brk: 0xbb2000) = 0xbb2000 1.201 ( 0.001 ms): usleep/26309 brk( ) = 0xbb2000 0.00user 0.00system 0:00.00elapsed 0%CPU (0avgtext+0avgdata 524maxresident)k 0inputs+0outputs (0major+180minor)pagefaults 0swaps [root@ssdandy ~]# BTW: to achieve the 'strace workload' behaviour, i.e. without a explicit '-f', one has to use --no-inherit. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> echo Link: http://lkml.kernel.org/n/tip-`ranpwd -l 24`@git.kernel.org Link: http://lkml.kernel.org/n/tip-2wu2d5n65msxoq1i7vtcaft2@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-13 12:32:45 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Disable events and drain events when forked workload ends We were not checking in the inner event processing loop if the forked workload had finished, which, on a busy system, may make it take a long time trying to drain events, entering a seemingly neverending loop, waiting for the system to get idle enough to make it drain the buffers. Fix it by disabling the events when 'done' is true, in the inner loop, to start draining what is in the buffers. Now: [root@ssdandy ~]# time trace --filter-pids 14003 -a sleep 1 | tail 996.748 ( 0.002 ms): sh/30296 rt_sigprocmask(how: SETMASK, nset: 0x7ffc83418160, sigsetsize: 8) = 0 996.751 ( 0.002 ms): sh/30296 rt_sigprocmask(how: BLOCK, nset: 0x7ffc834181f0, oset: 0x7ffc83418270, sigsetsize: 8) = 0 996.755 ( 0.002 ms): sh/30296 rt_sigaction(sig: INT, act: 0x7ffc83417f50, oact: 0x7ffc83417ff0, sigsetsize: 8) = 0 1004.543 ( 0.362 ms): tail/30198 ... [continued]: read()) = 4096 1004.548 ( 7.791 ms): sh/30296 wait4(upid: -1, stat_addr: 0x7ffc834181a0) ... 1004.975 ( 0.427 ms): tail/30198 read(buf: 0x7633f0, count: 8192) = 4096 1005.390 ( 0.410 ms): tail/30198 read(buf: 0x765410, count: 8192) = 4096 1005.743 ( 0.348 ms): tail/30198 read(buf: 0x7633f0, count: 8192) = 4096 1006.197 ( 0.449 ms): tail/30198 read(buf: 0x765410, count: 8192) = 4096 1006.492 ( 0.290 ms): tail/30198 read(buf: 0x7633f0, count: 8192) = 4096 real 0m1.219s user 0m0.704s sys 0m0.331s [root@ssdandy ~]# Reported-by: Michael Petlan <mpetlan@redhat.com> Suggested-by: Jiri Olsa <jolsa@redhat.com> Acked-by: Jiri Olsa <jolsa@redhat.com> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-p6kpn1b26qcbe47pufpw0tex@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-22 11:11:57 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Add summary option to dump syscall statistics When enabled dumps a summary of all syscalls by task with the usual statistics -- min, max, average and relative stddev. For example, make - 26341 : 3344 [ 17.4% ] 0.000 ms read : 52 0.000 4.802 0.644 30.08 write : 20 0.004 0.036 0.010 21.72 open : 24 0.003 0.046 0.014 23.68 close : 64 0.002 0.055 0.008 22.53 stat : 2714 0.002 0.222 0.004 4.47 fstat : 18 0.001 0.041 0.006 46.26 mmap : 30 0.003 0.009 0.006 5.71 mprotect : 8 0.006 0.039 0.016 32.16 munmap : 12 0.007 0.077 0.020 38.25 brk : 48 0.002 0.014 0.004 10.18 rt_sigaction : 18 0.002 0.002 0.002 2.11 rt_sigprocmask : 60 0.002 0.128 0.010 32.88 access : 2 0.006 0.006 0.006 0.00 pipe : 12 0.004 0.048 0.013 35.98 vfork : 34 0.448 0.980 0.692 3.04 execve : 20 0.000 0.387 0.046 56.66 wait4 : 34 0.017 9923.287 593.221 68.45 fcntl : 8 0.001 0.041 0.013 48.79 getdents : 48 0.002 0.079 0.013 19.62 getcwd : 2 0.005 0.005 0.005 0.00 chdir : 2 0.070 0.070 0.070 0.00 getrlimit : 2 0.045 0.045 0.045 0.00 arch_prctl : 2 0.002 0.002 0.002 0.00 setrlimit : 2 0.002 0.002 0.002 0.00 openat : 94 0.003 0.005 0.003 2.11 Signed-off-by: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung.kim@lge.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1381289214-24885-3-git-send-email-dsahern@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-08 21:26:53 -06:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Allow mixing with other events Basically adopting 'perf record' --event command line argument syntax: # trace -e \!mprotect,mmap,munmap,open,close,read,fstat,access,arch_prctl --event sched:*switch,sched:*exec,sched:*exit usleep 1 0.048 ( ): sched:sched_process_exec:filename=/bin/usleep pid=24732 old_pid=24732) 0.078 (0.002 ms): usleep/24732 brk( ) = 0x78f000 0.430 (0.002 ms): usleep/24732 brk( ) = 0x78f000 0.434 (0.003 ms): usleep/24732 brk(brk: 0x7b0000 ) = 0x7b0000 0.438 (0.001 ms): usleep/24732 brk( ) = 0x7b0000 0.460 (0.004 ms): usleep/24732 nanosleep(rqtp: 0x7ffff3696a40) ... 0.460 ( ): sched:sched_switch:prev_comm=usleep prev_pid=24732 prev_prio=120 prev_state=S ==> next_comm=swapper/1 next_pid=0 next_prio=120) 0.515 (0.058 ms): usleep/24732 ... [continued]: nanosleep()) = 0 0.520 (0.000 ms): usleep/24732 exit_group( 0.550 ( ): sched:sched_process_exit:comm=usleep pid=24732 prio=120) # Next steps, probably in this order: 1) Use ordered_events code, the logic in trace needs the events to be time ordered when needed, i.e. when multiple CPUs are involved. 2) Callchains! 3) Automatically account for interruptions when saying how long things took. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-gpst8mph575yb4wgf91qibyb@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-03 12:58:57 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Improve messages related to /proc/sys/kernel/perf_event_paranoid kernel/events/core.c has: /* * perf event paranoia level: * -1 - not paranoid at all * 0 - disallow raw tracepoint access for unpriv * 1 - disallow cpu events for unpriv * 2 - disallow kernel profiling for unpriv */ int sysctl_perf_event_paranoid __read_mostly = 1; So, with the default being 1, a non-root user can trace his stuff: [acme@zoo ~]$ cat /proc/sys/kernel/perf_event_paranoid 1 [acme@zoo ~]$ yes > /dev/null & [1] 15338 [acme@zoo ~]$ trace -p 15338 | head -5 0.005 ( 0.005 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 0.045 ( 0.001 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 0.085 ( 0.001 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 0.125 ( 0.001 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 0.165 ( 0.001 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 [acme@zoo ~]$ [acme@zoo ~]$ trace --duration 1 sleep 1 1002.148 (1001.218 ms): nanosleep(rqtp: 0x7fff46c79250 ) = 0 [acme@zoo ~]$ [acme@zoo ~]$ trace -- usleep 1 | tail -5 0.905 ( 0.002 ms): brk( ) = 0x1c82000 0.910 ( 0.003 ms): brk(brk: 0x1ca3000 ) = 0x1ca3000 0.913 ( 0.001 ms): brk( ) = 0x1ca3000 0.990 ( 0.059 ms): nanosleep(rqtp: 0x7fffe31a3280 ) = 0 0.995 ( 0.000 ms): exit_group( [acme@zoo ~]$ But can't do system wide tracing: [acme@zoo ~]$ trace Error: Operation not permitted. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 1. [acme@zoo ~]$ [acme@zoo ~]$ trace --cpu 0 Error: Operation not permitted. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 1. [acme@zoo ~]$ If the paranoid level is >= 2, i.e. turn this perf stuff off for !root users: [acme@zoo ~]$ sudo sh -c 'echo 2 > /proc/sys/kernel/perf_event_paranoid' [acme@zoo ~]$ cat /proc/sys/kernel/perf_event_paranoid 2 [acme@zoo ~]$ [acme@zoo ~]$ trace usleep 1 Error: Permission denied. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For your workloads it needs to be <= 1 Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 2. [acme@zoo ~]$ [acme@zoo ~]$ trace Error: Permission denied. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For your workloads it needs to be <= 1 Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 2. [acme@zoo ~]$ [acme@zoo ~]$ trace --cpu 1 Error: Permission denied. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For your workloads it needs to be <= 1 Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 2. [acme@zoo ~]$ If the user manages to get what he/she wants, convincing root not to be paranoid at all... [root@zoo ~]# echo -1 > /proc/sys/kernel/perf_event_paranoid [root@zoo ~]# cat /proc/sys/kernel/perf_event_paranoid -1 [root@zoo ~]# [acme@zoo ~]$ ps -eo user,pid,comm | grep Xorg root 729 Xorg [acme@zoo ~]$ [acme@zoo ~]$ trace -a --duration 0.001 -e \!select,ioctl,writev | grep Xorg | head -5 23.143 ( 0.003 ms): Xorg/729 setitimer(which: REAL, value: 0x7fffaadf16e0 ) = 0 23.152 ( 0.004 ms): Xorg/729 read(fd: 31, buf: 0x2544af0, count: 4096 ) = 8 23.161 ( 0.002 ms): Xorg/729 read(fd: 31, buf: 0x2544af0, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable 23.175 ( 0.002 ms): Xorg/729 setitimer(which: REAL, value: 0x7fffaadf16e0 ) = 0 23.235 ( 0.002 ms): Xorg/729 setitimer(which: REAL, value: 0x7fffaadf16e0 ) = 0 [acme@zoo ~]$ Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-di28olfwd28rvkox7v3hqhu1@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-17 17:38:29 -03:00
perf trace: Improve messages related to /proc/sys/kernel/perf_event_paranoid kernel/events/core.c has: /* * perf event paranoia level: * -1 - not paranoid at all * 0 - disallow raw tracepoint access for unpriv * 1 - disallow cpu events for unpriv * 2 - disallow kernel profiling for unpriv */ int sysctl_perf_event_paranoid __read_mostly = 1; So, with the default being 1, a non-root user can trace his stuff: [acme@zoo ~]$ cat /proc/sys/kernel/perf_event_paranoid 1 [acme@zoo ~]$ yes > /dev/null & [1] 15338 [acme@zoo ~]$ trace -p 15338 | head -5 0.005 ( 0.005 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 0.045 ( 0.001 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 0.085 ( 0.001 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 0.125 ( 0.001 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 0.165 ( 0.001 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 [acme@zoo ~]$ [acme@zoo ~]$ trace --duration 1 sleep 1 1002.148 (1001.218 ms): nanosleep(rqtp: 0x7fff46c79250 ) = 0 [acme@zoo ~]$ [acme@zoo ~]$ trace -- usleep 1 | tail -5 0.905 ( 0.002 ms): brk( ) = 0x1c82000 0.910 ( 0.003 ms): brk(brk: 0x1ca3000 ) = 0x1ca3000 0.913 ( 0.001 ms): brk( ) = 0x1ca3000 0.990 ( 0.059 ms): nanosleep(rqtp: 0x7fffe31a3280 ) = 0 0.995 ( 0.000 ms): exit_group( [acme@zoo ~]$ But can't do system wide tracing: [acme@zoo ~]$ trace Error: Operation not permitted. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 1. [acme@zoo ~]$ [acme@zoo ~]$ trace --cpu 0 Error: Operation not permitted. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 1. [acme@zoo ~]$ If the paranoid level is >= 2, i.e. turn this perf stuff off for !root users: [acme@zoo ~]$ sudo sh -c 'echo 2 > /proc/sys/kernel/perf_event_paranoid' [acme@zoo ~]$ cat /proc/sys/kernel/perf_event_paranoid 2 [acme@zoo ~]$ [acme@zoo ~]$ trace usleep 1 Error: Permission denied. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For your workloads it needs to be <= 1 Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 2. [acme@zoo ~]$ [acme@zoo ~]$ trace Error: Permission denied. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For your workloads it needs to be <= 1 Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 2. [acme@zoo ~]$ [acme@zoo ~]$ trace --cpu 1 Error: Permission denied. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For your workloads it needs to be <= 1 Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 2. [acme@zoo ~]$ If the user manages to get what he/she wants, convincing root not to be paranoid at all... [root@zoo ~]# echo -1 > /proc/sys/kernel/perf_event_paranoid [root@zoo ~]# cat /proc/sys/kernel/perf_event_paranoid -1 [root@zoo ~]# [acme@zoo ~]$ ps -eo user,pid,comm | grep Xorg root 729 Xorg [acme@zoo ~]$ [acme@zoo ~]$ trace -a --duration 0.001 -e \!select,ioctl,writev | grep Xorg | head -5 23.143 ( 0.003 ms): Xorg/729 setitimer(which: REAL, value: 0x7fffaadf16e0 ) = 0 23.152 ( 0.004 ms): Xorg/729 read(fd: 31, buf: 0x2544af0, count: 4096 ) = 8 23.161 ( 0.002 ms): Xorg/729 read(fd: 31, buf: 0x2544af0, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable 23.175 ( 0.002 ms): Xorg/729 setitimer(which: REAL, value: 0x7fffaadf16e0 ) = 0 23.235 ( 0.002 ms): Xorg/729 setitimer(which: REAL, value: 0x7fffaadf16e0 ) = 0 [acme@zoo ~]$ Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-di28olfwd28rvkox7v3hqhu1@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-17 17:38:29 -03:00
perf trace: Improve messages related to /proc/sys/kernel/perf_event_paranoid kernel/events/core.c has: /* * perf event paranoia level: * -1 - not paranoid at all * 0 - disallow raw tracepoint access for unpriv * 1 - disallow cpu events for unpriv * 2 - disallow kernel profiling for unpriv */ int sysctl_perf_event_paranoid __read_mostly = 1; So, with the default being 1, a non-root user can trace his stuff: [acme@zoo ~]$ cat /proc/sys/kernel/perf_event_paranoid 1 [acme@zoo ~]$ yes > /dev/null & [1] 15338 [acme@zoo ~]$ trace -p 15338 | head -5 0.005 ( 0.005 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 0.045 ( 0.001 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 0.085 ( 0.001 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 0.125 ( 0.001 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 0.165 ( 0.001 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 [acme@zoo ~]$ [acme@zoo ~]$ trace --duration 1 sleep 1 1002.148 (1001.218 ms): nanosleep(rqtp: 0x7fff46c79250 ) = 0 [acme@zoo ~]$ [acme@zoo ~]$ trace -- usleep 1 | tail -5 0.905 ( 0.002 ms): brk( ) = 0x1c82000 0.910 ( 0.003 ms): brk(brk: 0x1ca3000 ) = 0x1ca3000 0.913 ( 0.001 ms): brk( ) = 0x1ca3000 0.990 ( 0.059 ms): nanosleep(rqtp: 0x7fffe31a3280 ) = 0 0.995 ( 0.000 ms): exit_group( [acme@zoo ~]$ But can't do system wide tracing: [acme@zoo ~]$ trace Error: Operation not permitted. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 1. [acme@zoo ~]$ [acme@zoo ~]$ trace --cpu 0 Error: Operation not permitted. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 1. [acme@zoo ~]$ If the paranoid level is >= 2, i.e. turn this perf stuff off for !root users: [acme@zoo ~]$ sudo sh -c 'echo 2 > /proc/sys/kernel/perf_event_paranoid' [acme@zoo ~]$ cat /proc/sys/kernel/perf_event_paranoid 2 [acme@zoo ~]$ [acme@zoo ~]$ trace usleep 1 Error: Permission denied. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For your workloads it needs to be <= 1 Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 2. [acme@zoo ~]$ [acme@zoo ~]$ trace Error: Permission denied. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For your workloads it needs to be <= 1 Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 2. [acme@zoo ~]$ [acme@zoo ~]$ trace --cpu 1 Error: Permission denied. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For your workloads it needs to be <= 1 Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 2. [acme@zoo ~]$ If the user manages to get what he/she wants, convincing root not to be paranoid at all... [root@zoo ~]# echo -1 > /proc/sys/kernel/perf_event_paranoid [root@zoo ~]# cat /proc/sys/kernel/perf_event_paranoid -1 [root@zoo ~]# [acme@zoo ~]$ ps -eo user,pid,comm | grep Xorg root 729 Xorg [acme@zoo ~]$ [acme@zoo ~]$ trace -a --duration 0.001 -e \!select,ioctl,writev | grep Xorg | head -5 23.143 ( 0.003 ms): Xorg/729 setitimer(which: REAL, value: 0x7fffaadf16e0 ) = 0 23.152 ( 0.004 ms): Xorg/729 read(fd: 31, buf: 0x2544af0, count: 4096 ) = 8 23.161 ( 0.002 ms): Xorg/729 read(fd: 31, buf: 0x2544af0, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable 23.175 ( 0.002 ms): Xorg/729 setitimer(which: REAL, value: 0x7fffaadf16e0 ) = 0 23.235 ( 0.002 ms): Xorg/729 setitimer(which: REAL, value: 0x7fffaadf16e0 ) = 0 [acme@zoo ~]$ Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-di28olfwd28rvkox7v3hqhu1@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-17 17:38:29 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Use event filters for the event qualifier list We use raw_syscalls:sys_{enter,exit} events to show the syscalls, but were using a rather lazy/inneficient way to implement our 'strace -e' equivalent: filter out after reading the events in the ring buffer. Deflect more work to the kernel by appending a filter expression for that, that, together with the pid list, that is always present, if only to filter the tracer itself, reduces pressure on the ring buffer and otherwise use infrastructure already in place in the kernel to do early filtering. If we use it with -v we can see the filter passed to the kernel, for instance, for this contrieved case: # trace -v -e \!open,close,write,poll,recvfrom,select,recvmsg,writev,sendmsg,read,futex,epoll_wait,ioctl,eventfd --filter-pids 2189,2566,1398,2692,4475,4532 <SNIP> (common_pid != 2514 && common_pid != 1398 && common_pid != 2189 && common_pid != 2566 && common_pid != 2692 && common_pid != 4475 && common_pid != 4532) && (id != 3 && id != 232 && id != 284 && id != 202 && id != 16 && id != 2 && id != 7 && id != 0 && id != 45 && id != 47 && id != 23 && id != 46 && id != 1 && id != 20) 0.011 (0.011 ms): caribou/2295 eventfd2(flags: CLOEXEC|NONBLOCK) = 18 16.946 (0.019 ms): caribou/2295 eventfd2(flags: CLOEXEC|NONBLOCK) = 18 38.598 (0.167 ms): chronyd/794 socket(family: INET, type: DGRAM ) = 4 38.603 (0.002 ms): chronyd/794 fcntl(fd: 4<socket:[239307]>, cmd: GETFD) = 0 38.605 (0.001 ms): chronyd/794 fcntl(fd: 4<socket:[239307]>, cmd: SETFD, arg: 1) = 0 ^C # Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-ti2tg18atproqpguc2moinp6@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-07-04 12:44:59 -03:00
perf trace: Improve messages related to /proc/sys/kernel/perf_event_paranoid kernel/events/core.c has: /* * perf event paranoia level: * -1 - not paranoid at all * 0 - disallow raw tracepoint access for unpriv * 1 - disallow cpu events for unpriv * 2 - disallow kernel profiling for unpriv */ int sysctl_perf_event_paranoid __read_mostly = 1; So, with the default being 1, a non-root user can trace his stuff: [acme@zoo ~]$ cat /proc/sys/kernel/perf_event_paranoid 1 [acme@zoo ~]$ yes > /dev/null & [1] 15338 [acme@zoo ~]$ trace -p 15338 | head -5 0.005 ( 0.005 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 0.045 ( 0.001 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 0.085 ( 0.001 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 0.125 ( 0.001 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 0.165 ( 0.001 ms): write(fd: 1</dev/null>, buf: 0x7fe6db765000, count: 4096 ) = 4096 [acme@zoo ~]$ [acme@zoo ~]$ trace --duration 1 sleep 1 1002.148 (1001.218 ms): nanosleep(rqtp: 0x7fff46c79250 ) = 0 [acme@zoo ~]$ [acme@zoo ~]$ trace -- usleep 1 | tail -5 0.905 ( 0.002 ms): brk( ) = 0x1c82000 0.910 ( 0.003 ms): brk(brk: 0x1ca3000 ) = 0x1ca3000 0.913 ( 0.001 ms): brk( ) = 0x1ca3000 0.990 ( 0.059 ms): nanosleep(rqtp: 0x7fffe31a3280 ) = 0 0.995 ( 0.000 ms): exit_group( [acme@zoo ~]$ But can't do system wide tracing: [acme@zoo ~]$ trace Error: Operation not permitted. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 1. [acme@zoo ~]$ [acme@zoo ~]$ trace --cpu 0 Error: Operation not permitted. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 1. [acme@zoo ~]$ If the paranoid level is >= 2, i.e. turn this perf stuff off for !root users: [acme@zoo ~]$ sudo sh -c 'echo 2 > /proc/sys/kernel/perf_event_paranoid' [acme@zoo ~]$ cat /proc/sys/kernel/perf_event_paranoid 2 [acme@zoo ~]$ [acme@zoo ~]$ trace usleep 1 Error: Permission denied. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For your workloads it needs to be <= 1 Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 2. [acme@zoo ~]$ [acme@zoo ~]$ trace Error: Permission denied. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For your workloads it needs to be <= 1 Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 2. [acme@zoo ~]$ [acme@zoo ~]$ trace --cpu 1 Error: Permission denied. Hint: Check /proc/sys/kernel/perf_event_paranoid setting. Hint: For your workloads it needs to be <= 1 Hint: For system wide tracing it needs to be set to -1. Hint: The current value is 2. [acme@zoo ~]$ If the user manages to get what he/she wants, convincing root not to be paranoid at all... [root@zoo ~]# echo -1 > /proc/sys/kernel/perf_event_paranoid [root@zoo ~]# cat /proc/sys/kernel/perf_event_paranoid -1 [root@zoo ~]# [acme@zoo ~]$ ps -eo user,pid,comm | grep Xorg root 729 Xorg [acme@zoo ~]$ [acme@zoo ~]$ trace -a --duration 0.001 -e \!select,ioctl,writev | grep Xorg | head -5 23.143 ( 0.003 ms): Xorg/729 setitimer(which: REAL, value: 0x7fffaadf16e0 ) = 0 23.152 ( 0.004 ms): Xorg/729 read(fd: 31, buf: 0x2544af0, count: 4096 ) = 8 23.161 ( 0.002 ms): Xorg/729 read(fd: 31, buf: 0x2544af0, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable 23.175 ( 0.002 ms): Xorg/729 setitimer(which: REAL, value: 0x7fffaadf16e0 ) = 0 23.235 ( 0.002 ms): Xorg/729 setitimer(which: REAL, value: 0x7fffaadf16e0 ) = 0 [acme@zoo ~]$ Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-di28olfwd28rvkox7v3hqhu1@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-17 17:38:29 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Support using -f to override perf.data file ownership Enable perf trace to use perf.data when it is not owned by current user or root. Example: # perf trace record ls # chown Yunlong.Song:Yunlong.Song perf.data # ls -al perf.data -rw------- 1 Yunlong.Song Yunlong.Song 4153101 Apr 2 15:28 perf.data # id uid=0(root) gid=0(root) groups=0(root),64(pkcs11) Before this patch: # perf trace -i perf.data File perf.data not owned by current user or root (use -f to override) # perf trace -i perf.data -f Error: unknown switch `f' usage: perf trace [<options>] [<command>] or: perf trace [<options>] -- <command> [<options>] or: perf trace record [<options>] [<command>] or: perf trace record [<options>] -- <command> [<options>] --event <event> event selector. use 'perf list' to list available events --comm show the thread COMM next to its id --tool_stats show tool stats -e, --expr <expr> list of events to trace -o, --output <file> output file name -i, --input <file> Analyze events in file -p, --pid <pid> trace events on existing process id -t, --tid <tid> trace events on existing thread id --filter-pids <float> ... As shown above, the -f option does not work at all. After this patch: # perf trace -i perf.data File perf.data not owned by current user or root (use -f to override) # perf trace -i perf.data -f 0.056 ( 0.002 ms): ls/47325 brk( ... 0.108 ( 0.018 ms): ls/47325 mmap(len: 4096, prot: READ|WRITE, ... 0.145 ( 0.013 ms): ls/47325 access(filename: 0x7f31259a0eb0, ... 0.172 ( 0.008 ms): ls/47325 open(filename: 0x7fffeb9a0d00, ... 0.180 ( 0.004 ms): ls/47325 stat(filename: 0x7fffeb9a0d00, ... 0.185 ( 0.004 ms): ls/47325 open(filename: 0x7fffeb9a0d00, ... 0.189 ( 0.003 ms): ls/47325 stat(filename: 0x7fffeb9a0d00, ... 0.195 ( 0.004 ms): ls/47325 open(filename: 0x7fffeb9a0d00, ... 0.199 ( 0.002 ms): ls/47325 stat(filename: 0x7fffeb9a0d00, ... 0.205 ( 0.004 ms): ls/47325 open(filename: 0x7fffeb9a0d00, ... 0.211 ( 0.004 ms): ls/47325 stat(filename: 0x7fffeb9a0d00, ... 0.220 ( 0.007 ms): ls/47325 open(filename: 0x7f312599e8ff, ... ... ... As shown above, the -f option really works now. Signed-off-by: Yunlong Song <yunlong.song@huawei.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Wang Nan <wangnan0@huawei.com> Link: http://lkml.kernel.org/r/1427982439-27388-10-git-send-email-yunlong.song@huawei.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-02 21:47:18 +08:00
perf tools: Check recorded kernel version when finding vmlinux Currently vmlinux_path__init() only tries to find vmlinux file from current directory, /boot and some canonical directories with version number of the running kernel. This can be a problem when reporting old data recorded on a kernel version not running currently. We can use --symfs option for this but it's annoying for user to do it always. As we already have the info in the perf.data file, it can be changed to use it for the search automatically. Before: $ perf report ... # Samples: 4K of event 'cpu-clock' # Event count (approx.): 1067250000 # # Overhead Command Shared Object Symbol # ........ .......... ................. .............................. 71.87% swapper [kernel.kallsyms] [k] recover_probed_instruction After: # Overhead Command Shared Object Symbol # ........ .......... ................. .................... 71.87% swapper [kernel.kallsyms] [k] native_safe_halt This requires to change signature of symbol__init() to receive struct perf_session_env *. Reported-by: Minchan Kim <minchan@kernel.org> Signed-off-by: Namhyung Kim <namhyung@kernel.org> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com> Cc: Minchan Kim <minchan@kernel.org> Cc: Namhyung Kim <namhyung.kim@lge.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1407825645-24586-14-git-send-email-namhyung@kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-08-12 15:40:45 +09:00
perf trace: Add possibility to switch off syscall events Currently, we may either trace syscalls or syscalls+pagefaults. We'd like to be able to trace *only* pagefaults and this commit implements this feature. Example: [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace --no-syscalls -F -p `pidof xchat` 0.000 ( 0.000 ms): xchat/4574 majfault [g_unichar_get_script+0x11] => /usr/lib64/libglib-2.0.so.0.3800.2@0xc403b (x.) 0.202 ( 0.000 ms): xchat/4574 majfault [_cairo_hash_table_lookup+0x53] => 0x2280ff0 (?.) 20.854 ( 0.000 ms): xchat/4574 majfault [gdk_cairo_set_source_pixbuf+0x110] => /usr/bin/xchat@0x6da1f (x.) 1022.000 ( 0.000 ms): xchat/4574 majfault [__memcpy_sse2_unaligned+0x29] => 0x7ff5a8ca0400 (?.) ^C[root@zoo /]# Below we can see malloc calls, 'trace' reading symbol tables in libraries to resolve symbols, etc. [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace --no-syscalls -F all --cpu 1 sleep 10 0.000 ( 0.000 ms): chrome/26589 minfault [0x1b53129] => /tmp/perf-26589.map@0x33cbcbf7f000 (x.) 96.477 ( 0.000 ms): libvirtd/947 minfault [copy_user_enhanced_fast_string+0x5] => 0x7f7685bba000 (?k) 113.164 ( 0.000 ms): Xorg/1063 minfault [0x786da] => 0x7fce52882a3c (?.) 7162.801 ( 0.000 ms): chrome/3747 minfault [0x8e1a89] => 0xfcaefed0008 (?.) <SNIP> 7773.138 ( 0.000 ms): chrome/3886 minfault [0x8e1a89] => 0xfcb0ce28008 (?.) 7992.022 ( 0.000 ms): chrome/26574 minfault [0x1b5a708] => 0x3de7b5fc5000 (?.) 8108.949 ( 0.000 ms): qemu-system-x8/4537 majfault [_int_malloc+0xee] => 0x7faffc466d60 (?.) 8108.975 ( 0.000 ms): qemu-system-x8/4537 minfault [_int_malloc+0x102] => 0x7faffc466d60 (?.) <SNIP> 8148.174 ( 0.000 ms): qemu-system-x8/4537 minfault [_int_malloc+0x102] => 0x7faffc4eb500 (?.) 8270.855 ( 0.000 ms): chrome/26245 minfault [do_bo_emit_reloc+0xdb] => 0x45d092bc004 (?.) 8270.869 ( 0.000 ms): chrome/26245 minfault [do_bo_emit_reloc+0x108] => 0x45d09150000 (?.) no symbols found in /usr/lib64/libspice-server.so.1.9.0, maybe install a debug package? 8273.831 ( 0.000 ms): trace/20198 majfault [__memcmp_sse4_1+0xbc6] => /usr/lib64/libspice-server.so.1.9.0@0xdf000 (d.) <SNIP> 8275.121 ( 0.000 ms): trace/20198 minfault [dso__load+0x38] => 0x14fe756 (?.) no symbols found in /usr/lib64/libelf-0.158.so, maybe install a debug package? 8275.142 ( 0.000 ms): trace/20198 minfault [__memcmp_sse4_1+0xbc6] => /usr/lib64/libelf-0.158.so@0x0 (d.) <SNIP> [root@zoo /]# Signed-off-by: Stanislav Fomichev <stfomichev@yandex-team.ru> Cc: David Ahern <dsahern@gmail.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1403799268-1367-6-git-send-email-stfomichev@yandex-team.ru Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-06-26 20:14:28 +04:00
perf trace: Add possibility to switch off syscall events Currently, we may either trace syscalls or syscalls+pagefaults. We'd like to be able to trace *only* pagefaults and this commit implements this feature. Example: [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace --no-syscalls -F -p `pidof xchat` 0.000 ( 0.000 ms): xchat/4574 majfault [g_unichar_get_script+0x11] => /usr/lib64/libglib-2.0.so.0.3800.2@0xc403b (x.) 0.202 ( 0.000 ms): xchat/4574 majfault [_cairo_hash_table_lookup+0x53] => 0x2280ff0 (?.) 20.854 ( 0.000 ms): xchat/4574 majfault [gdk_cairo_set_source_pixbuf+0x110] => /usr/bin/xchat@0x6da1f (x.) 1022.000 ( 0.000 ms): xchat/4574 majfault [__memcpy_sse2_unaligned+0x29] => 0x7ff5a8ca0400 (?.) ^C[root@zoo /]# Below we can see malloc calls, 'trace' reading symbol tables in libraries to resolve symbols, etc. [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace --no-syscalls -F all --cpu 1 sleep 10 0.000 ( 0.000 ms): chrome/26589 minfault [0x1b53129] => /tmp/perf-26589.map@0x33cbcbf7f000 (x.) 96.477 ( 0.000 ms): libvirtd/947 minfault [copy_user_enhanced_fast_string+0x5] => 0x7f7685bba000 (?k) 113.164 ( 0.000 ms): Xorg/1063 minfault [0x786da] => 0x7fce52882a3c (?.) 7162.801 ( 0.000 ms): chrome/3747 minfault [0x8e1a89] => 0xfcaefed0008 (?.) <SNIP> 7773.138 ( 0.000 ms): chrome/3886 minfault [0x8e1a89] => 0xfcb0ce28008 (?.) 7992.022 ( 0.000 ms): chrome/26574 minfault [0x1b5a708] => 0x3de7b5fc5000 (?.) 8108.949 ( 0.000 ms): qemu-system-x8/4537 majfault [_int_malloc+0xee] => 0x7faffc466d60 (?.) 8108.975 ( 0.000 ms): qemu-system-x8/4537 minfault [_int_malloc+0x102] => 0x7faffc466d60 (?.) <SNIP> 8148.174 ( 0.000 ms): qemu-system-x8/4537 minfault [_int_malloc+0x102] => 0x7faffc4eb500 (?.) 8270.855 ( 0.000 ms): chrome/26245 minfault [do_bo_emit_reloc+0xdb] => 0x45d092bc004 (?.) 8270.869 ( 0.000 ms): chrome/26245 minfault [do_bo_emit_reloc+0x108] => 0x45d09150000 (?.) no symbols found in /usr/lib64/libspice-server.so.1.9.0, maybe install a debug package? 8273.831 ( 0.000 ms): trace/20198 majfault [__memcmp_sse4_1+0xbc6] => /usr/lib64/libspice-server.so.1.9.0@0xdf000 (d.) <SNIP> 8275.121 ( 0.000 ms): trace/20198 minfault [dso__load+0x38] => 0x14fe756 (?.) no symbols found in /usr/lib64/libelf-0.158.so, maybe install a debug package? 8275.142 ( 0.000 ms): trace/20198 minfault [__memcmp_sse4_1+0xbc6] => /usr/lib64/libelf-0.158.so@0x0 (d.) <SNIP> [root@zoo /]# Signed-off-by: Stanislav Fomichev <stfomichev@yandex-team.ru> Cc: David Ahern <dsahern@gmail.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1403799268-1367-6-git-send-email-stfomichev@yandex-team.ru Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-06-26 20:14:28 +04:00
perf trace: Add pagefaults record and replay support Previous commit added live pagefault trace support, this one adds record and replay support. Example: [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace -F all record -a sleep 10 [ perf record: Woken up 0 times to write data ] [ perf record: Captured and wrote 1029.722 MB perf.data (~44989242 samples) ] [root@zoo /]# ls -la perf.data -rw-------. 1 root root 1083921722 Jun 26 17:44 perf.data [root@zoo /]# perf evlist raw_syscalls:sys_enter raw_syscalls:sys_exit major-faults minor-faults [root@zoo /]# trace -i perf.data | grep -v trace\/ | tail -15 156.137 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.139 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) 156.140 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0xc4] => 0x0 (?.) 156.144 ( 0.000 ms): perl/18476 minfault [_int_free+0xda] => 0x0 (?.) 156.151 ( 0.000 ms): perl/18476 minfault [_int_free+0x1df] => 0x0 (?.) 156.158 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.161 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.168 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.172 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.173 ( 0.000 ms): perl/18476 minfault [_int_free+0xda] => 0x0 (?.) 156.183 ( 0.000 ms): perl/18476 minfault [Perl_hfree_next_entry+0xb4] => 0x0 (?.) 156.197 ( 0.000 ms): perl/18476 minfault [_int_free+0x1df] => 0x0 (?.) 156.216 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) 156.221 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) [root@zoo /]# Signed-off-by: Stanislav Fomichev <stfomichev@yandex-team.ru> Cc: David Ahern <dsahern@gmail.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1403799268-1367-4-git-send-email-stfomichev@yandex-team.ru Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-06-26 20:14:26 +04:00
perf trace: Add summary option to dump syscall statistics When enabled dumps a summary of all syscalls by task with the usual statistics -- min, max, average and relative stddev. For example, make - 26341 : 3344 [ 17.4% ] 0.000 ms read : 52 0.000 4.802 0.644 30.08 write : 20 0.004 0.036 0.010 21.72 open : 24 0.003 0.046 0.014 23.68 close : 64 0.002 0.055 0.008 22.53 stat : 2714 0.002 0.222 0.004 4.47 fstat : 18 0.001 0.041 0.006 46.26 mmap : 30 0.003 0.009 0.006 5.71 mprotect : 8 0.006 0.039 0.016 32.16 munmap : 12 0.007 0.077 0.020 38.25 brk : 48 0.002 0.014 0.004 10.18 rt_sigaction : 18 0.002 0.002 0.002 2.11 rt_sigprocmask : 60 0.002 0.128 0.010 32.88 access : 2 0.006 0.006 0.006 0.00 pipe : 12 0.004 0.048 0.013 35.98 vfork : 34 0.448 0.980 0.692 3.04 execve : 20 0.000 0.387 0.046 56.66 wait4 : 34 0.017 9923.287 593.221 68.45 fcntl : 8 0.001 0.041 0.013 48.79 getdents : 48 0.002 0.079 0.013 19.62 getcwd : 2 0.005 0.005 0.005 0.00 chdir : 2 0.070 0.070 0.070 0.00 getrlimit : 2 0.045 0.045 0.045 0.00 arch_prctl : 2 0.002 0.002 0.002 0.00 setrlimit : 2 0.002 0.002 0.002 0.00 openat : 94 0.003 0.005 0.003 2.11 Signed-off-by: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung.kim@lge.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1381289214-24885-3-git-send-email-dsahern@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-08 21:26:53 -06:00
perf trace: Use sched:sched_stat_runtime to provide a thread summary [root@sandy ~]# perf trace --sched --duration 0.100 --pid `pidof firefox` <SNIP> 17079.847 ( 0.009 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.892 ( 0.010 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable 17079.921 ( 0.013 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.949 ( 0.009 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ firefox - 17643 : 18013 [ 72.2% ] 359.110 ms firefox - 17663 : 41 [ 0.2% ] 21.439 ms firefox - 17664 : 6840 [ 27.4% ] 133.642 ms firefox - 17667 : 46 [ 0.2% ] 0.682 ms [root@sandy ~]# This is equivalent to the 'perf trace summary' subcomand in the tmp.perf/trace2 branch. Another example, setting a huge duration filter to get just a system wide summary: [root@sandy ~]# perf trace --duration 10000.0 --sched ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ scsi_eh_1 - 258 : 15 [ 0.0% ] 0.133 ms kworker/0:1H - 322 : 13 [ 0.0% ] 0.032 ms jbd2/dm-0-8 - 384 : 4 [ 0.0% ] 0.115 ms flush-253:0 - 470 : 1 [ 0.0% ] 0.027 ms firefox - 950 : 4783 [ 0.1% ] 24.863 ms firefox - 992 : 1883 [ 0.1% ] 6.808 ms firefox - 995 : 35 [ 0.0% ] 0.111 ms ksoftirqd/6 - 4362 : 2 [ 0.0% ] 0.005 ms ksoftirqd/7 - 4365 : 1 [ 0.0% ] 0.007 ms Xorg - 4671 : 148 [ 0.0% ] 0.912 ms gnome-settings- - 4846 : 14 [ 0.0% ] 0.086 ms seahorse-daemon - 4847 : 14 [ 0.0% ] 0.092 ms gnome-panel - 4875 : 46 [ 0.0% ] 0.159 ms gnome-power-man - 4918 : 16 [ 0.0% ] 0.065 ms gvfs-afc-volume - 4992 : 77 [ 0.0% ] 0.136 ms gnome-screensav - 5114 : 24 [ 0.0% ] 0.128 ms xchat - 8082 : 466 [ 0.0% ] 2.019 ms synergyc - 8369 : 941 [ 0.0% ] 3.291 ms synergyc - 8371 : 85 [ 0.0% ] 1.817 ms jbd2/dm-4-8 - 9352 : 4 [ 0.0% ] 0.109 ms rpcbind - 9786 : 3 [ 0.0% ] 0.017 ms rtkit-daemon - 12802 : 10 [ 0.0% ] 0.038 ms rtkit-daemon - 12803 : 8 [ 0.0% ] 0.000 ms udisks-daemon - 13020 : 27 [ 0.0% ] 0.240 ms kworker/7:0 - 14651 : 669 [ 0.0% ] 2.616 ms kworker/5:1 - 16220 : 2 [ 0.0% ] 0.069 ms kworker/4:0 - 19776 : 13 [ 0.0% ] 0.176 ms openvpn - 20131 : 133 [ 0.0% ] 0.762 ms plugin-containe - 20508 : 60658 [ 1.7% ] 131.153 ms npviewer.bin - 20520 : 72208 [ 2.0% ] 138.945 ms npviewer.bin - 20542 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 20543 : 30 [ 0.0% ] 0.074 ms npviewer.bin - 20547 : 35 [ 0.0% ] 0.092 ms npviewer.bin - 20552 : 35 [ 0.0% ] 0.093 ms sshd - 20645 : 32 [ 0.0% ] 0.071 ms npviewer.bin - 21053 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 21054 : 35 [ 0.0% ] 0.097 ms kworker/0:2 - 21169 : 149 [ 0.0% ] 1.143 ms kworker/3:0 - 22171 : 113 [ 0.0% ] 96.892 ms flush-253:4 - 22410 : 1 [ 0.0% ] 0.028 ms kworker/6:0 - 24581 : 25 [ 0.0% ] 0.275 ms kworker/1:0 - 25572 : 4 [ 0.0% ] 0.103 ms kworker/2:1 - 26299 : 138 [ 0.0% ] 1.440 ms kworker/0:0 - 26325 : 1 [ 0.0% ] 0.003 ms perf - 26330 : 3506967 [ 96.1% ] 6648.310 ms [root@sandy ~]# Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-mzuli0srnxyi1o029py6537x@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-17 17:13:12 -03:00
perf trace: Add summary option to dump syscall statistics When enabled dumps a summary of all syscalls by task with the usual statistics -- min, max, average and relative stddev. For example, make - 26341 : 3344 [ 17.4% ] 0.000 ms read : 52 0.000 4.802 0.644 30.08 write : 20 0.004 0.036 0.010 21.72 open : 24 0.003 0.046 0.014 23.68 close : 64 0.002 0.055 0.008 22.53 stat : 2714 0.002 0.222 0.004 4.47 fstat : 18 0.001 0.041 0.006 46.26 mmap : 30 0.003 0.009 0.006 5.71 mprotect : 8 0.006 0.039 0.016 32.16 munmap : 12 0.007 0.077 0.020 38.25 brk : 48 0.002 0.014 0.004 10.18 rt_sigaction : 18 0.002 0.002 0.002 2.11 rt_sigprocmask : 60 0.002 0.128 0.010 32.88 access : 2 0.006 0.006 0.006 0.00 pipe : 12 0.004 0.048 0.013 35.98 vfork : 34 0.448 0.980 0.692 3.04 execve : 20 0.000 0.387 0.046 56.66 wait4 : 34 0.017 9923.287 593.221 68.45 fcntl : 8 0.001 0.041 0.013 48.79 getdents : 48 0.002 0.079 0.013 19.62 getcwd : 2 0.005 0.005 0.005 0.00 chdir : 2 0.070 0.070 0.070 0.00 getrlimit : 2 0.045 0.045 0.045 0.00 arch_prctl : 2 0.002 0.002 0.002 0.00 setrlimit : 2 0.002 0.002 0.002 0.00 openat : 94 0.003 0.005 0.003 2.11 Signed-off-by: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung.kim@lge.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1381289214-24885-3-git-send-email-dsahern@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-08 21:26:53 -06:00
perf trace: Add total time column to summary. It is cumbersome to manually calculate the total time spent in a given syscall by multiplying the average value with the number of calls. Instead, we now do this directly inside perf trace. Note that this is also done by 'strace', which even adds a column with relative numbers - something we could do in the future. Example: perf trace -s find /some/folder > /dev/null Summary of events: find (19976), 700123 events, 100.0%, 0.000 msec syscall calls total min avg max stddev (msec) (msec) (msec) (msec) (%) --------------- -------- --------- --------- --------- --------- ------ read 4 0.006 0.001 0.002 0.003 27.42% write 8046 9.617 0.001 0.001 0.035 0.56% open 34196 40.384 0.001 0.001 0.071 0.30% close 68375 57.104 0.001 0.001 0.076 0.25% stat 4 0.004 0.001 0.001 0.001 3.14% fstat 34189 27.518 0.001 0.001 0.060 0.34% mmap 13 0.029 0.001 0.002 0.003 10.74% mprotect 6 0.018 0.002 0.003 0.005 17.04% munmap 3 0.014 0.003 0.005 0.006 24.87% brk 87 0.490 0.001 0.006 0.016 6.50% ioctl 3 0.004 0.001 0.001 0.003 36.39% access 1 0.004 0.004 0.004 0.004 0.00% uname 1 0.001 0.001 0.001 0.001 0.00% getdents 68393 143.600 0.001 0.002 0.187 0.95% fchdir 68371 56.980 0.001 0.001 0.111 0.39% arch_prctl 1 0.001 0.001 0.001 0.001 0.00% openat 34184 41.737 0.001 0.001 0.102 0.41% newfstatat 34184 41.180 0.001 0.001 0.064 0.34% Signed-off-by: Milian Wolff <milian.wolff@kdab.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> LPU-Reference: 1438853069-5902-1-git-send-email-milian.wolff@kdab.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-08-06 11:24:29 +02:00
perf trace: Add summary option to dump syscall statistics When enabled dumps a summary of all syscalls by task with the usual statistics -- min, max, average and relative stddev. For example, make - 26341 : 3344 [ 17.4% ] 0.000 ms read : 52 0.000 4.802 0.644 30.08 write : 20 0.004 0.036 0.010 21.72 open : 24 0.003 0.046 0.014 23.68 close : 64 0.002 0.055 0.008 22.53 stat : 2714 0.002 0.222 0.004 4.47 fstat : 18 0.001 0.041 0.006 46.26 mmap : 30 0.003 0.009 0.006 5.71 mprotect : 8 0.006 0.039 0.016 32.16 munmap : 12 0.007 0.077 0.020 38.25 brk : 48 0.002 0.014 0.004 10.18 rt_sigaction : 18 0.002 0.002 0.002 2.11 rt_sigprocmask : 60 0.002 0.128 0.010 32.88 access : 2 0.006 0.006 0.006 0.00 pipe : 12 0.004 0.048 0.013 35.98 vfork : 34 0.448 0.980 0.692 3.04 execve : 20 0.000 0.387 0.046 56.66 wait4 : 34 0.017 9923.287 593.221 68.45 fcntl : 8 0.001 0.041 0.013 48.79 getdents : 48 0.002 0.079 0.013 19.62 getcwd : 2 0.005 0.005 0.005 0.00 chdir : 2 0.070 0.070 0.070 0.00 getrlimit : 2 0.045 0.045 0.045 0.00 arch_prctl : 2 0.002 0.002 0.002 0.00 setrlimit : 2 0.002 0.002 0.002 0.00 openat : 94 0.003 0.005 0.003 2.11 Signed-off-by: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung.kim@lge.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1381289214-24885-3-git-send-email-dsahern@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-08 21:26:53 -06:00
perf trace: Add total time column to summary. It is cumbersome to manually calculate the total time spent in a given syscall by multiplying the average value with the number of calls. Instead, we now do this directly inside perf trace. Note that this is also done by 'strace', which even adds a column with relative numbers - something we could do in the future. Example: perf trace -s find /some/folder > /dev/null Summary of events: find (19976), 700123 events, 100.0%, 0.000 msec syscall calls total min avg max stddev (msec) (msec) (msec) (msec) (%) --------------- -------- --------- --------- --------- --------- ------ read 4 0.006 0.001 0.002 0.003 27.42% write 8046 9.617 0.001 0.001 0.035 0.56% open 34196 40.384 0.001 0.001 0.071 0.30% close 68375 57.104 0.001 0.001 0.076 0.25% stat 4 0.004 0.001 0.001 0.001 3.14% fstat 34189 27.518 0.001 0.001 0.060 0.34% mmap 13 0.029 0.001 0.002 0.003 10.74% mprotect 6 0.018 0.002 0.003 0.005 17.04% munmap 3 0.014 0.003 0.005 0.006 24.87% brk 87 0.490 0.001 0.006 0.016 6.50% ioctl 3 0.004 0.001 0.001 0.003 36.39% access 1 0.004 0.004 0.004 0.004 0.00% uname 1 0.001 0.001 0.001 0.001 0.00% getdents 68393 143.600 0.001 0.002 0.187 0.95% fchdir 68371 56.980 0.001 0.001 0.111 0.39% arch_prctl 1 0.001 0.001 0.001 0.001 0.00% openat 34184 41.737 0.001 0.001 0.102 0.41% newfstatat 34184 41.180 0.001 0.001 0.064 0.34% Signed-off-by: Milian Wolff <milian.wolff@kdab.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> LPU-Reference: 1438853069-5902-1-git-send-email-milian.wolff@kdab.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-08-06 11:24:29 +02:00
perf trace: Add summary option to dump syscall statistics When enabled dumps a summary of all syscalls by task with the usual statistics -- min, max, average and relative stddev. For example, make - 26341 : 3344 [ 17.4% ] 0.000 ms read : 52 0.000 4.802 0.644 30.08 write : 20 0.004 0.036 0.010 21.72 open : 24 0.003 0.046 0.014 23.68 close : 64 0.002 0.055 0.008 22.53 stat : 2714 0.002 0.222 0.004 4.47 fstat : 18 0.001 0.041 0.006 46.26 mmap : 30 0.003 0.009 0.006 5.71 mprotect : 8 0.006 0.039 0.016 32.16 munmap : 12 0.007 0.077 0.020 38.25 brk : 48 0.002 0.014 0.004 10.18 rt_sigaction : 18 0.002 0.002 0.002 2.11 rt_sigprocmask : 60 0.002 0.128 0.010 32.88 access : 2 0.006 0.006 0.006 0.00 pipe : 12 0.004 0.048 0.013 35.98 vfork : 34 0.448 0.980 0.692 3.04 execve : 20 0.000 0.387 0.046 56.66 wait4 : 34 0.017 9923.287 593.221 68.45 fcntl : 8 0.001 0.041 0.013 48.79 getdents : 48 0.002 0.079 0.013 19.62 getcwd : 2 0.005 0.005 0.005 0.00 chdir : 2 0.070 0.070 0.070 0.00 getrlimit : 2 0.045 0.045 0.045 0.00 arch_prctl : 2 0.002 0.002 0.002 0.00 setrlimit : 2 0.002 0.002 0.002 0.00 openat : 94 0.003 0.005 0.003 2.11 Signed-off-by: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung.kim@lge.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1381289214-24885-3-git-send-email-dsahern@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-08 21:26:53 -06:00
perf trace: Use sched:sched_stat_runtime to provide a thread summary [root@sandy ~]# perf trace --sched --duration 0.100 --pid `pidof firefox` <SNIP> 17079.847 ( 0.009 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.892 ( 0.010 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable 17079.921 ( 0.013 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.949 ( 0.009 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ firefox - 17643 : 18013 [ 72.2% ] 359.110 ms firefox - 17663 : 41 [ 0.2% ] 21.439 ms firefox - 17664 : 6840 [ 27.4% ] 133.642 ms firefox - 17667 : 46 [ 0.2% ] 0.682 ms [root@sandy ~]# This is equivalent to the 'perf trace summary' subcomand in the tmp.perf/trace2 branch. Another example, setting a huge duration filter to get just a system wide summary: [root@sandy ~]# perf trace --duration 10000.0 --sched ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ scsi_eh_1 - 258 : 15 [ 0.0% ] 0.133 ms kworker/0:1H - 322 : 13 [ 0.0% ] 0.032 ms jbd2/dm-0-8 - 384 : 4 [ 0.0% ] 0.115 ms flush-253:0 - 470 : 1 [ 0.0% ] 0.027 ms firefox - 950 : 4783 [ 0.1% ] 24.863 ms firefox - 992 : 1883 [ 0.1% ] 6.808 ms firefox - 995 : 35 [ 0.0% ] 0.111 ms ksoftirqd/6 - 4362 : 2 [ 0.0% ] 0.005 ms ksoftirqd/7 - 4365 : 1 [ 0.0% ] 0.007 ms Xorg - 4671 : 148 [ 0.0% ] 0.912 ms gnome-settings- - 4846 : 14 [ 0.0% ] 0.086 ms seahorse-daemon - 4847 : 14 [ 0.0% ] 0.092 ms gnome-panel - 4875 : 46 [ 0.0% ] 0.159 ms gnome-power-man - 4918 : 16 [ 0.0% ] 0.065 ms gvfs-afc-volume - 4992 : 77 [ 0.0% ] 0.136 ms gnome-screensav - 5114 : 24 [ 0.0% ] 0.128 ms xchat - 8082 : 466 [ 0.0% ] 2.019 ms synergyc - 8369 : 941 [ 0.0% ] 3.291 ms synergyc - 8371 : 85 [ 0.0% ] 1.817 ms jbd2/dm-4-8 - 9352 : 4 [ 0.0% ] 0.109 ms rpcbind - 9786 : 3 [ 0.0% ] 0.017 ms rtkit-daemon - 12802 : 10 [ 0.0% ] 0.038 ms rtkit-daemon - 12803 : 8 [ 0.0% ] 0.000 ms udisks-daemon - 13020 : 27 [ 0.0% ] 0.240 ms kworker/7:0 - 14651 : 669 [ 0.0% ] 2.616 ms kworker/5:1 - 16220 : 2 [ 0.0% ] 0.069 ms kworker/4:0 - 19776 : 13 [ 0.0% ] 0.176 ms openvpn - 20131 : 133 [ 0.0% ] 0.762 ms plugin-containe - 20508 : 60658 [ 1.7% ] 131.153 ms npviewer.bin - 20520 : 72208 [ 2.0% ] 138.945 ms npviewer.bin - 20542 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 20543 : 30 [ 0.0% ] 0.074 ms npviewer.bin - 20547 : 35 [ 0.0% ] 0.092 ms npviewer.bin - 20552 : 35 [ 0.0% ] 0.093 ms sshd - 20645 : 32 [ 0.0% ] 0.071 ms npviewer.bin - 21053 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 21054 : 35 [ 0.0% ] 0.097 ms kworker/0:2 - 21169 : 149 [ 0.0% ] 1.143 ms kworker/3:0 - 22171 : 113 [ 0.0% ] 96.892 ms flush-253:4 - 22410 : 1 [ 0.0% ] 0.028 ms kworker/6:0 - 24581 : 25 [ 0.0% ] 0.275 ms kworker/1:0 - 25572 : 4 [ 0.0% ] 0.103 ms kworker/2:1 - 26299 : 138 [ 0.0% ] 1.440 ms kworker/0:0 - 26325 : 1 [ 0.0% ] 0.003 ms perf - 26330 : 3506967 [ 96.1% ] 6648.310 ms [root@sandy ~]# Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-mzuli0srnxyi1o029py6537x@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-17 17:13:12 -03:00
perf trace: Add summary option to dump syscall statistics When enabled dumps a summary of all syscalls by task with the usual statistics -- min, max, average and relative stddev. For example, make - 26341 : 3344 [ 17.4% ] 0.000 ms read : 52 0.000 4.802 0.644 30.08 write : 20 0.004 0.036 0.010 21.72 open : 24 0.003 0.046 0.014 23.68 close : 64 0.002 0.055 0.008 22.53 stat : 2714 0.002 0.222 0.004 4.47 fstat : 18 0.001 0.041 0.006 46.26 mmap : 30 0.003 0.009 0.006 5.71 mprotect : 8 0.006 0.039 0.016 32.16 munmap : 12 0.007 0.077 0.020 38.25 brk : 48 0.002 0.014 0.004 10.18 rt_sigaction : 18 0.002 0.002 0.002 2.11 rt_sigprocmask : 60 0.002 0.128 0.010 32.88 access : 2 0.006 0.006 0.006 0.00 pipe : 12 0.004 0.048 0.013 35.98 vfork : 34 0.448 0.980 0.692 3.04 execve : 20 0.000 0.387 0.046 56.66 wait4 : 34 0.017 9923.287 593.221 68.45 fcntl : 8 0.001 0.041 0.013 48.79 getdents : 48 0.002 0.079 0.013 19.62 getcwd : 2 0.005 0.005 0.005 0.00 chdir : 2 0.070 0.070 0.070 0.00 getrlimit : 2 0.045 0.045 0.045 0.00 arch_prctl : 2 0.002 0.002 0.002 0.00 setrlimit : 2 0.002 0.002 0.002 0.00 openat : 94 0.003 0.005 0.003 2.11 Signed-off-by: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Ingo Molnar <mingo@kernel.org> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung.kim@lge.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1381289214-24885-3-git-send-email-dsahern@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-10-08 21:26:53 -06:00
perf trace: Use sched:sched_stat_runtime to provide a thread summary [root@sandy ~]# perf trace --sched --duration 0.100 --pid `pidof firefox` <SNIP> 17079.847 ( 0.009 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.892 ( 0.010 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable 17079.921 ( 0.013 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.949 ( 0.009 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ firefox - 17643 : 18013 [ 72.2% ] 359.110 ms firefox - 17663 : 41 [ 0.2% ] 21.439 ms firefox - 17664 : 6840 [ 27.4% ] 133.642 ms firefox - 17667 : 46 [ 0.2% ] 0.682 ms [root@sandy ~]# This is equivalent to the 'perf trace summary' subcomand in the tmp.perf/trace2 branch. Another example, setting a huge duration filter to get just a system wide summary: [root@sandy ~]# perf trace --duration 10000.0 --sched ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ scsi_eh_1 - 258 : 15 [ 0.0% ] 0.133 ms kworker/0:1H - 322 : 13 [ 0.0% ] 0.032 ms jbd2/dm-0-8 - 384 : 4 [ 0.0% ] 0.115 ms flush-253:0 - 470 : 1 [ 0.0% ] 0.027 ms firefox - 950 : 4783 [ 0.1% ] 24.863 ms firefox - 992 : 1883 [ 0.1% ] 6.808 ms firefox - 995 : 35 [ 0.0% ] 0.111 ms ksoftirqd/6 - 4362 : 2 [ 0.0% ] 0.005 ms ksoftirqd/7 - 4365 : 1 [ 0.0% ] 0.007 ms Xorg - 4671 : 148 [ 0.0% ] 0.912 ms gnome-settings- - 4846 : 14 [ 0.0% ] 0.086 ms seahorse-daemon - 4847 : 14 [ 0.0% ] 0.092 ms gnome-panel - 4875 : 46 [ 0.0% ] 0.159 ms gnome-power-man - 4918 : 16 [ 0.0% ] 0.065 ms gvfs-afc-volume - 4992 : 77 [ 0.0% ] 0.136 ms gnome-screensav - 5114 : 24 [ 0.0% ] 0.128 ms xchat - 8082 : 466 [ 0.0% ] 2.019 ms synergyc - 8369 : 941 [ 0.0% ] 3.291 ms synergyc - 8371 : 85 [ 0.0% ] 1.817 ms jbd2/dm-4-8 - 9352 : 4 [ 0.0% ] 0.109 ms rpcbind - 9786 : 3 [ 0.0% ] 0.017 ms rtkit-daemon - 12802 : 10 [ 0.0% ] 0.038 ms rtkit-daemon - 12803 : 8 [ 0.0% ] 0.000 ms udisks-daemon - 13020 : 27 [ 0.0% ] 0.240 ms kworker/7:0 - 14651 : 669 [ 0.0% ] 2.616 ms kworker/5:1 - 16220 : 2 [ 0.0% ] 0.069 ms kworker/4:0 - 19776 : 13 [ 0.0% ] 0.176 ms openvpn - 20131 : 133 [ 0.0% ] 0.762 ms plugin-containe - 20508 : 60658 [ 1.7% ] 131.153 ms npviewer.bin - 20520 : 72208 [ 2.0% ] 138.945 ms npviewer.bin - 20542 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 20543 : 30 [ 0.0% ] 0.074 ms npviewer.bin - 20547 : 35 [ 0.0% ] 0.092 ms npviewer.bin - 20552 : 35 [ 0.0% ] 0.093 ms sshd - 20645 : 32 [ 0.0% ] 0.071 ms npviewer.bin - 21053 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 21054 : 35 [ 0.0% ] 0.097 ms kworker/0:2 - 21169 : 149 [ 0.0% ] 1.143 ms kworker/3:0 - 22171 : 113 [ 0.0% ] 96.892 ms flush-253:4 - 22410 : 1 [ 0.0% ] 0.028 ms kworker/6:0 - 24581 : 25 [ 0.0% ] 0.275 ms kworker/1:0 - 25572 : 4 [ 0.0% ] 0.103 ms kworker/2:1 - 26299 : 138 [ 0.0% ] 1.440 ms kworker/0:0 - 26325 : 1 [ 0.0% ] 0.003 ms perf - 26330 : 3506967 [ 96.1% ] 6648.310 ms [root@sandy ~]# Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-mzuli0srnxyi1o029py6537x@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-17 17:13:12 -03:00
perf trace: Use sched:sched_stat_runtime to provide a thread summary [root@sandy ~]# perf trace --sched --duration 0.100 --pid `pidof firefox` <SNIP> 17079.847 ( 0.009 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.892 ( 0.010 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable 17079.921 ( 0.013 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.949 ( 0.009 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ firefox - 17643 : 18013 [ 72.2% ] 359.110 ms firefox - 17663 : 41 [ 0.2% ] 21.439 ms firefox - 17664 : 6840 [ 27.4% ] 133.642 ms firefox - 17667 : 46 [ 0.2% ] 0.682 ms [root@sandy ~]# This is equivalent to the 'perf trace summary' subcomand in the tmp.perf/trace2 branch. Another example, setting a huge duration filter to get just a system wide summary: [root@sandy ~]# perf trace --duration 10000.0 --sched ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ scsi_eh_1 - 258 : 15 [ 0.0% ] 0.133 ms kworker/0:1H - 322 : 13 [ 0.0% ] 0.032 ms jbd2/dm-0-8 - 384 : 4 [ 0.0% ] 0.115 ms flush-253:0 - 470 : 1 [ 0.0% ] 0.027 ms firefox - 950 : 4783 [ 0.1% ] 24.863 ms firefox - 992 : 1883 [ 0.1% ] 6.808 ms firefox - 995 : 35 [ 0.0% ] 0.111 ms ksoftirqd/6 - 4362 : 2 [ 0.0% ] 0.005 ms ksoftirqd/7 - 4365 : 1 [ 0.0% ] 0.007 ms Xorg - 4671 : 148 [ 0.0% ] 0.912 ms gnome-settings- - 4846 : 14 [ 0.0% ] 0.086 ms seahorse-daemon - 4847 : 14 [ 0.0% ] 0.092 ms gnome-panel - 4875 : 46 [ 0.0% ] 0.159 ms gnome-power-man - 4918 : 16 [ 0.0% ] 0.065 ms gvfs-afc-volume - 4992 : 77 [ 0.0% ] 0.136 ms gnome-screensav - 5114 : 24 [ 0.0% ] 0.128 ms xchat - 8082 : 466 [ 0.0% ] 2.019 ms synergyc - 8369 : 941 [ 0.0% ] 3.291 ms synergyc - 8371 : 85 [ 0.0% ] 1.817 ms jbd2/dm-4-8 - 9352 : 4 [ 0.0% ] 0.109 ms rpcbind - 9786 : 3 [ 0.0% ] 0.017 ms rtkit-daemon - 12802 : 10 [ 0.0% ] 0.038 ms rtkit-daemon - 12803 : 8 [ 0.0% ] 0.000 ms udisks-daemon - 13020 : 27 [ 0.0% ] 0.240 ms kworker/7:0 - 14651 : 669 [ 0.0% ] 2.616 ms kworker/5:1 - 16220 : 2 [ 0.0% ] 0.069 ms kworker/4:0 - 19776 : 13 [ 0.0% ] 0.176 ms openvpn - 20131 : 133 [ 0.0% ] 0.762 ms plugin-containe - 20508 : 60658 [ 1.7% ] 131.153 ms npviewer.bin - 20520 : 72208 [ 2.0% ] 138.945 ms npviewer.bin - 20542 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 20543 : 30 [ 0.0% ] 0.074 ms npviewer.bin - 20547 : 35 [ 0.0% ] 0.092 ms npviewer.bin - 20552 : 35 [ 0.0% ] 0.093 ms sshd - 20645 : 32 [ 0.0% ] 0.071 ms npviewer.bin - 21053 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 21054 : 35 [ 0.0% ] 0.097 ms kworker/0:2 - 21169 : 149 [ 0.0% ] 1.143 ms kworker/3:0 - 22171 : 113 [ 0.0% ] 96.892 ms flush-253:4 - 22410 : 1 [ 0.0% ] 0.028 ms kworker/6:0 - 24581 : 25 [ 0.0% ] 0.275 ms kworker/1:0 - 25572 : 4 [ 0.0% ] 0.103 ms kworker/2:1 - 26299 : 138 [ 0.0% ] 1.440 ms kworker/0:0 - 26325 : 1 [ 0.0% ] 0.003 ms perf - 26330 : 3506967 [ 96.1% ] 6648.310 ms [root@sandy ~]# Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-mzuli0srnxyi1o029py6537x@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-17 17:13:12 -03:00
perf trace: Use sched:sched_stat_runtime to provide a thread summary [root@sandy ~]# perf trace --sched --duration 0.100 --pid `pidof firefox` <SNIP> 17079.847 ( 0.009 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.892 ( 0.010 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable 17079.921 ( 0.013 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.949 ( 0.009 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ firefox - 17643 : 18013 [ 72.2% ] 359.110 ms firefox - 17663 : 41 [ 0.2% ] 21.439 ms firefox - 17664 : 6840 [ 27.4% ] 133.642 ms firefox - 17667 : 46 [ 0.2% ] 0.682 ms [root@sandy ~]# This is equivalent to the 'perf trace summary' subcomand in the tmp.perf/trace2 branch. Another example, setting a huge duration filter to get just a system wide summary: [root@sandy ~]# perf trace --duration 10000.0 --sched ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ scsi_eh_1 - 258 : 15 [ 0.0% ] 0.133 ms kworker/0:1H - 322 : 13 [ 0.0% ] 0.032 ms jbd2/dm-0-8 - 384 : 4 [ 0.0% ] 0.115 ms flush-253:0 - 470 : 1 [ 0.0% ] 0.027 ms firefox - 950 : 4783 [ 0.1% ] 24.863 ms firefox - 992 : 1883 [ 0.1% ] 6.808 ms firefox - 995 : 35 [ 0.0% ] 0.111 ms ksoftirqd/6 - 4362 : 2 [ 0.0% ] 0.005 ms ksoftirqd/7 - 4365 : 1 [ 0.0% ] 0.007 ms Xorg - 4671 : 148 [ 0.0% ] 0.912 ms gnome-settings- - 4846 : 14 [ 0.0% ] 0.086 ms seahorse-daemon - 4847 : 14 [ 0.0% ] 0.092 ms gnome-panel - 4875 : 46 [ 0.0% ] 0.159 ms gnome-power-man - 4918 : 16 [ 0.0% ] 0.065 ms gvfs-afc-volume - 4992 : 77 [ 0.0% ] 0.136 ms gnome-screensav - 5114 : 24 [ 0.0% ] 0.128 ms xchat - 8082 : 466 [ 0.0% ] 2.019 ms synergyc - 8369 : 941 [ 0.0% ] 3.291 ms synergyc - 8371 : 85 [ 0.0% ] 1.817 ms jbd2/dm-4-8 - 9352 : 4 [ 0.0% ] 0.109 ms rpcbind - 9786 : 3 [ 0.0% ] 0.017 ms rtkit-daemon - 12802 : 10 [ 0.0% ] 0.038 ms rtkit-daemon - 12803 : 8 [ 0.0% ] 0.000 ms udisks-daemon - 13020 : 27 [ 0.0% ] 0.240 ms kworker/7:0 - 14651 : 669 [ 0.0% ] 2.616 ms kworker/5:1 - 16220 : 2 [ 0.0% ] 0.069 ms kworker/4:0 - 19776 : 13 [ 0.0% ] 0.176 ms openvpn - 20131 : 133 [ 0.0% ] 0.762 ms plugin-containe - 20508 : 60658 [ 1.7% ] 131.153 ms npviewer.bin - 20520 : 72208 [ 2.0% ] 138.945 ms npviewer.bin - 20542 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 20543 : 30 [ 0.0% ] 0.074 ms npviewer.bin - 20547 : 35 [ 0.0% ] 0.092 ms npviewer.bin - 20552 : 35 [ 0.0% ] 0.093 ms sshd - 20645 : 32 [ 0.0% ] 0.071 ms npviewer.bin - 21053 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 21054 : 35 [ 0.0% ] 0.097 ms kworker/0:2 - 21169 : 149 [ 0.0% ] 1.143 ms kworker/3:0 - 22171 : 113 [ 0.0% ] 96.892 ms flush-253:4 - 22410 : 1 [ 0.0% ] 0.028 ms kworker/6:0 - 24581 : 25 [ 0.0% ] 0.275 ms kworker/1:0 - 25572 : 4 [ 0.0% ] 0.103 ms kworker/2:1 - 26299 : 138 [ 0.0% ] 1.440 ms kworker/0:0 - 26325 : 1 [ 0.0% ] 0.003 ms perf - 26330 : 3506967 [ 96.1% ] 6648.310 ms [root@sandy ~]# Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-mzuli0srnxyi1o029py6537x@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-17 17:13:12 -03:00
perf trace: Use sched:sched_stat_runtime to provide a thread summary [root@sandy ~]# perf trace --sched --duration 0.100 --pid `pidof firefox` <SNIP> 17079.847 ( 0.009 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.892 ( 0.010 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable 17079.921 ( 0.013 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.949 ( 0.009 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ firefox - 17643 : 18013 [ 72.2% ] 359.110 ms firefox - 17663 : 41 [ 0.2% ] 21.439 ms firefox - 17664 : 6840 [ 27.4% ] 133.642 ms firefox - 17667 : 46 [ 0.2% ] 0.682 ms [root@sandy ~]# This is equivalent to the 'perf trace summary' subcomand in the tmp.perf/trace2 branch. Another example, setting a huge duration filter to get just a system wide summary: [root@sandy ~]# perf trace --duration 10000.0 --sched ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ scsi_eh_1 - 258 : 15 [ 0.0% ] 0.133 ms kworker/0:1H - 322 : 13 [ 0.0% ] 0.032 ms jbd2/dm-0-8 - 384 : 4 [ 0.0% ] 0.115 ms flush-253:0 - 470 : 1 [ 0.0% ] 0.027 ms firefox - 950 : 4783 [ 0.1% ] 24.863 ms firefox - 992 : 1883 [ 0.1% ] 6.808 ms firefox - 995 : 35 [ 0.0% ] 0.111 ms ksoftirqd/6 - 4362 : 2 [ 0.0% ] 0.005 ms ksoftirqd/7 - 4365 : 1 [ 0.0% ] 0.007 ms Xorg - 4671 : 148 [ 0.0% ] 0.912 ms gnome-settings- - 4846 : 14 [ 0.0% ] 0.086 ms seahorse-daemon - 4847 : 14 [ 0.0% ] 0.092 ms gnome-panel - 4875 : 46 [ 0.0% ] 0.159 ms gnome-power-man - 4918 : 16 [ 0.0% ] 0.065 ms gvfs-afc-volume - 4992 : 77 [ 0.0% ] 0.136 ms gnome-screensav - 5114 : 24 [ 0.0% ] 0.128 ms xchat - 8082 : 466 [ 0.0% ] 2.019 ms synergyc - 8369 : 941 [ 0.0% ] 3.291 ms synergyc - 8371 : 85 [ 0.0% ] 1.817 ms jbd2/dm-4-8 - 9352 : 4 [ 0.0% ] 0.109 ms rpcbind - 9786 : 3 [ 0.0% ] 0.017 ms rtkit-daemon - 12802 : 10 [ 0.0% ] 0.038 ms rtkit-daemon - 12803 : 8 [ 0.0% ] 0.000 ms udisks-daemon - 13020 : 27 [ 0.0% ] 0.240 ms kworker/7:0 - 14651 : 669 [ 0.0% ] 2.616 ms kworker/5:1 - 16220 : 2 [ 0.0% ] 0.069 ms kworker/4:0 - 19776 : 13 [ 0.0% ] 0.176 ms openvpn - 20131 : 133 [ 0.0% ] 0.762 ms plugin-containe - 20508 : 60658 [ 1.7% ] 131.153 ms npviewer.bin - 20520 : 72208 [ 2.0% ] 138.945 ms npviewer.bin - 20542 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 20543 : 30 [ 0.0% ] 0.074 ms npviewer.bin - 20547 : 35 [ 0.0% ] 0.092 ms npviewer.bin - 20552 : 35 [ 0.0% ] 0.093 ms sshd - 20645 : 32 [ 0.0% ] 0.071 ms npviewer.bin - 21053 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 21054 : 35 [ 0.0% ] 0.097 ms kworker/0:2 - 21169 : 149 [ 0.0% ] 1.143 ms kworker/3:0 - 22171 : 113 [ 0.0% ] 96.892 ms flush-253:4 - 22410 : 1 [ 0.0% ] 0.028 ms kworker/6:0 - 24581 : 25 [ 0.0% ] 0.275 ms kworker/1:0 - 25572 : 4 [ 0.0% ] 0.103 ms kworker/2:1 - 26299 : 138 [ 0.0% ] 1.440 ms kworker/0:0 - 26325 : 1 [ 0.0% ] 0.003 ms perf - 26330 : 3506967 [ 96.1% ] 6648.310 ms [root@sandy ~]# Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-mzuli0srnxyi1o029py6537x@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-17 17:13:12 -03:00
perf trace: Introduce --filter-pids When tracing in X we get event loops due to the tracing activity, i.e. updates to a gnome-terminal that generate syscalls for X.org, etc. To get a more useful view of what is happening, syscall wise, system wide, we need to filter those, like in: # ps ax|egrep '981|2296|1519' | grep -v egrep 981 tty1 Ss+ 5:40 /usr/bin/Xorg :0 -background none ... 1519 ? Sl 2:22 /usr/bin/gnome-shell 2296 ? Sl 4:16 /usr/libexec/gnome-terminal-server # # trace -e write --filter-pids 981,2296,1519 0.385 ( 0.021 ms): goa-daemon/2061 write(fd: 1</dev/null>, buf: 0x7fbeb017b000, count: 136) = 136 0.922 ( 0.014 ms): goa-daemon/2061 write(fd: 1</dev/null>, buf: 0x7fbeb017b000, count: 140) = 140 5006.525 ( 0.029 ms): goa-daemon/2061 write(fd: 1</dev/null>, buf: 0x7fbeb017b000, count: 136) = 136 5007.235 ( 0.023 ms): goa-daemon/2061 write(fd: 1</dev/null>, buf: 0x7fbeb017b000, count: 140) = 140 5177.646 ( 0.018 ms): rtkit-daemon/782 write(fd: 5<anon_inode:[eventfd]>, buf: 0x7f7eea70be88, count: 8) = 8 8314.497 ( 0.004 ms): gsd-locate-poi/2084 write(fd: 5<anon_inode:[eventfd]>, buf: 0x7fffe96af7b0, count: 8) = 8 8314.518 ( 0.002 ms): gsd-locate-poi/2084 write(fd: 5<anon_inode:[eventfd]>, buf: 0x7fffe96af0e0, count: 8) = 8 ^C# When this option is used the tracer pid is also filtered. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-f5qmiyy7c0uxdm21ncatpeek@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-21 11:36:52 -08:00
perf trace: Allow mixing with other events Basically adopting 'perf record' --event command line argument syntax: # trace -e \!mprotect,mmap,munmap,open,close,read,fstat,access,arch_prctl --event sched:*switch,sched:*exec,sched:*exit usleep 1 0.048 ( ): sched:sched_process_exec:filename=/bin/usleep pid=24732 old_pid=24732) 0.078 (0.002 ms): usleep/24732 brk( ) = 0x78f000 0.430 (0.002 ms): usleep/24732 brk( ) = 0x78f000 0.434 (0.003 ms): usleep/24732 brk(brk: 0x7b0000 ) = 0x7b0000 0.438 (0.001 ms): usleep/24732 brk( ) = 0x7b0000 0.460 (0.004 ms): usleep/24732 nanosleep(rqtp: 0x7ffff3696a40) ... 0.460 ( ): sched:sched_switch:prev_comm=usleep prev_pid=24732 prev_prio=120 prev_state=S ==> next_comm=swapper/1 next_pid=0 next_prio=120) 0.515 (0.058 ms): usleep/24732 ... [continued]: nanosleep()) = 0 0.520 (0.000 ms): usleep/24732 exit_group( 0.550 ( ): sched:sched_process_exit:comm=usleep pid=24732 prio=120) # Next steps, probably in this order: 1) Use ordered_events code, the logic in trace needs the events to be time ordered when needed, i.e. when multiple CPUs are involved. 2) Callchains! 3) Automatically account for interruptions when saying how long things took. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-gpst8mph575yb4wgf91qibyb@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-03 12:58:57 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Add option to show process COMM Enabled by default, disable with --no-comm, e.g.: 181.821 (0.001 ms): deja-dup-monit/10784 recvmsg(fd: 8, msg: 0x7fff4342baf0, flags: PEEK|TRUNC|CMSG_CLOEXEC ) = 20 181.824 (0.001 ms): deja-dup-monit/10784 geteuid( ) = 1000 181.825 (0.001 ms): deja-dup-monit/10784 getegid( ) = 1000 181.834 (0.002 ms): deja-dup-monit/10784 recvmsg(fd: 8, msg: 0x7fff4342baf0, flags: CMSG_CLOEXEC ) = 20 181.836 (0.001 ms): deja-dup-monit/10784 geteuid( ) = 1000 181.838 (0.001 ms): deja-dup-monit/10784 getegid( ) = 1000 181.705 (0.003 ms): evolution-addr/10924 recvmsg(fd: 10, msg: 0x7fff17dc6990, flags: PEEK|TRUNC|CMSG_CLOEXEC) = 1256 181.710 (0.002 ms): evolution-addr/10924 geteuid( ) = 1000 181.712 (0.001 ms): evolution-addr/10924 getegid( ) = 1000 181.727 (0.003 ms): evolution-addr/10924 recvmsg(fd: 10, msg: 0x7fff17dc6990, flags: CMSG_CLOEXEC ) = 1256 181.731 (0.001 ms): evolution-addr/10924 geteuid( ) = 1000 181.734 (0.001 ms): evolution-addr/10924 getegid( ) = 1000 181.908 (0.002 ms): evolution-addr/10924 recvmsg(fd: 10, msg: 0x7fff17dc6990, flags: PEEK|TRUNC|CMSG_CLOEXEC) = 20 181.913 (0.001 ms): evolution-addr/10924 geteuid( ) = 1000 181.915 (0.001 ms): evolution-addr/10924 getegid( ) = 1000 181.930 (0.003 ms): evolution-addr/10924 recvmsg(fd: 10, msg: 0x7fff17dc6990, flags: CMSG_CLOEXEC ) = 20 181.934 (0.001 ms): evolution-addr/10924 geteuid( ) = 1000 181.937 (0.001 ms): evolution-addr/10924 getegid( ) = 1000 220.718 (0.010 ms): at-spi2-regist/10715 sendmsg(fd: 3, msg: 0x7fffdb8756c0, flags: NOSIGNAL ) = 200 220.741 (0.000 ms): dbus-daemon/10711 ... [continued]: epoll_wait()) = 1 220.759 (0.004 ms): dbus-daemon/10711 recvmsg(fd: 11, msg: 0x7ffff94594d0, flags: CMSG_CLOEXEC ) = 200 220.780 (0.002 ms): dbus-daemon/10711 recvmsg(fd: 11, msg: 0x7ffff94594d0, flags: CMSG_CLOEXEC ) = 200 220.788 (0.001 ms): dbus-daemon/10711 recvmsg(fd: 11, msg: 0x7ffff94594d0, flags: CMSG_CLOEXEC ) = -1 EAGAIN Resource temporarily unavailable 220.760 (0.004 ms): at-spi2-regist/10715 sendmsg(fd: 3, msg: 0x7fffdb8756c0, flags: NOSIGNAL ) = 200 220.771 (0.023 ms): perf/26347 open(filename: 0xf2e780, mode: 15918976 ) = 19 220.850 (0.002 ms): perf/26347 close(fd: 19 ) = 0 Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-6be5jvnkdzjptdrebfn5263n@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-09-12 12:35:21 -03:00
perf trace: Add possibility to switch off syscall events Currently, we may either trace syscalls or syscalls+pagefaults. We'd like to be able to trace *only* pagefaults and this commit implements this feature. Example: [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace --no-syscalls -F -p `pidof xchat` 0.000 ( 0.000 ms): xchat/4574 majfault [g_unichar_get_script+0x11] => /usr/lib64/libglib-2.0.so.0.3800.2@0xc403b (x.) 0.202 ( 0.000 ms): xchat/4574 majfault [_cairo_hash_table_lookup+0x53] => 0x2280ff0 (?.) 20.854 ( 0.000 ms): xchat/4574 majfault [gdk_cairo_set_source_pixbuf+0x110] => /usr/bin/xchat@0x6da1f (x.) 1022.000 ( 0.000 ms): xchat/4574 majfault [__memcpy_sse2_unaligned+0x29] => 0x7ff5a8ca0400 (?.) ^C[root@zoo /]# Below we can see malloc calls, 'trace' reading symbol tables in libraries to resolve symbols, etc. [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace --no-syscalls -F all --cpu 1 sleep 10 0.000 ( 0.000 ms): chrome/26589 minfault [0x1b53129] => /tmp/perf-26589.map@0x33cbcbf7f000 (x.) 96.477 ( 0.000 ms): libvirtd/947 minfault [copy_user_enhanced_fast_string+0x5] => 0x7f7685bba000 (?k) 113.164 ( 0.000 ms): Xorg/1063 minfault [0x786da] => 0x7fce52882a3c (?.) 7162.801 ( 0.000 ms): chrome/3747 minfault [0x8e1a89] => 0xfcaefed0008 (?.) <SNIP> 7773.138 ( 0.000 ms): chrome/3886 minfault [0x8e1a89] => 0xfcb0ce28008 (?.) 7992.022 ( 0.000 ms): chrome/26574 minfault [0x1b5a708] => 0x3de7b5fc5000 (?.) 8108.949 ( 0.000 ms): qemu-system-x8/4537 majfault [_int_malloc+0xee] => 0x7faffc466d60 (?.) 8108.975 ( 0.000 ms): qemu-system-x8/4537 minfault [_int_malloc+0x102] => 0x7faffc466d60 (?.) <SNIP> 8148.174 ( 0.000 ms): qemu-system-x8/4537 minfault [_int_malloc+0x102] => 0x7faffc4eb500 (?.) 8270.855 ( 0.000 ms): chrome/26245 minfault [do_bo_emit_reloc+0xdb] => 0x45d092bc004 (?.) 8270.869 ( 0.000 ms): chrome/26245 minfault [do_bo_emit_reloc+0x108] => 0x45d09150000 (?.) no symbols found in /usr/lib64/libspice-server.so.1.9.0, maybe install a debug package? 8273.831 ( 0.000 ms): trace/20198 majfault [__memcmp_sse4_1+0xbc6] => /usr/lib64/libspice-server.so.1.9.0@0xdf000 (d.) <SNIP> 8275.121 ( 0.000 ms): trace/20198 minfault [dso__load+0x38] => 0x14fe756 (?.) no symbols found in /usr/lib64/libelf-0.158.so, maybe install a debug package? 8275.142 ( 0.000 ms): trace/20198 minfault [__memcmp_sse4_1+0xbc6] => /usr/lib64/libelf-0.158.so@0x0 (d.) <SNIP> [root@zoo /]# Signed-off-by: Stanislav Fomichev <stfomichev@yandex-team.ru> Cc: David Ahern <dsahern@gmail.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1403799268-1367-6-git-send-email-stfomichev@yandex-team.ru Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-06-26 20:14:28 +04:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Allow mixing with other events Basically adopting 'perf record' --event command line argument syntax: # trace -e \!mprotect,mmap,munmap,open,close,read,fstat,access,arch_prctl --event sched:*switch,sched:*exec,sched:*exit usleep 1 0.048 ( ): sched:sched_process_exec:filename=/bin/usleep pid=24732 old_pid=24732) 0.078 (0.002 ms): usleep/24732 brk( ) = 0x78f000 0.430 (0.002 ms): usleep/24732 brk( ) = 0x78f000 0.434 (0.003 ms): usleep/24732 brk(brk: 0x7b0000 ) = 0x7b0000 0.438 (0.001 ms): usleep/24732 brk( ) = 0x7b0000 0.460 (0.004 ms): usleep/24732 nanosleep(rqtp: 0x7ffff3696a40) ... 0.460 ( ): sched:sched_switch:prev_comm=usleep prev_pid=24732 prev_prio=120 prev_state=S ==> next_comm=swapper/1 next_pid=0 next_prio=120) 0.515 (0.058 ms): usleep/24732 ... [continued]: nanosleep()) = 0 0.520 (0.000 ms): usleep/24732 exit_group( 0.550 ( ): sched:sched_process_exit:comm=usleep pid=24732 prio=120) # Next steps, probably in this order: 1) Use ordered_events code, the logic in trace needs the events to be time ordered when needed, i.e. when multiple CPUs are involved. 2) Callchains! 3) Automatically account for interruptions when saying how long things took. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-gpst8mph575yb4wgf91qibyb@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-03 12:58:57 -03:00
perf trace: Add option to show process COMM Enabled by default, disable with --no-comm, e.g.: 181.821 (0.001 ms): deja-dup-monit/10784 recvmsg(fd: 8, msg: 0x7fff4342baf0, flags: PEEK|TRUNC|CMSG_CLOEXEC ) = 20 181.824 (0.001 ms): deja-dup-monit/10784 geteuid( ) = 1000 181.825 (0.001 ms): deja-dup-monit/10784 getegid( ) = 1000 181.834 (0.002 ms): deja-dup-monit/10784 recvmsg(fd: 8, msg: 0x7fff4342baf0, flags: CMSG_CLOEXEC ) = 20 181.836 (0.001 ms): deja-dup-monit/10784 geteuid( ) = 1000 181.838 (0.001 ms): deja-dup-monit/10784 getegid( ) = 1000 181.705 (0.003 ms): evolution-addr/10924 recvmsg(fd: 10, msg: 0x7fff17dc6990, flags: PEEK|TRUNC|CMSG_CLOEXEC) = 1256 181.710 (0.002 ms): evolution-addr/10924 geteuid( ) = 1000 181.712 (0.001 ms): evolution-addr/10924 getegid( ) = 1000 181.727 (0.003 ms): evolution-addr/10924 recvmsg(fd: 10, msg: 0x7fff17dc6990, flags: CMSG_CLOEXEC ) = 1256 181.731 (0.001 ms): evolution-addr/10924 geteuid( ) = 1000 181.734 (0.001 ms): evolution-addr/10924 getegid( ) = 1000 181.908 (0.002 ms): evolution-addr/10924 recvmsg(fd: 10, msg: 0x7fff17dc6990, flags: PEEK|TRUNC|CMSG_CLOEXEC) = 20 181.913 (0.001 ms): evolution-addr/10924 geteuid( ) = 1000 181.915 (0.001 ms): evolution-addr/10924 getegid( ) = 1000 181.930 (0.003 ms): evolution-addr/10924 recvmsg(fd: 10, msg: 0x7fff17dc6990, flags: CMSG_CLOEXEC ) = 20 181.934 (0.001 ms): evolution-addr/10924 geteuid( ) = 1000 181.937 (0.001 ms): evolution-addr/10924 getegid( ) = 1000 220.718 (0.010 ms): at-spi2-regist/10715 sendmsg(fd: 3, msg: 0x7fffdb8756c0, flags: NOSIGNAL ) = 200 220.741 (0.000 ms): dbus-daemon/10711 ... [continued]: epoll_wait()) = 1 220.759 (0.004 ms): dbus-daemon/10711 recvmsg(fd: 11, msg: 0x7ffff94594d0, flags: CMSG_CLOEXEC ) = 200 220.780 (0.002 ms): dbus-daemon/10711 recvmsg(fd: 11, msg: 0x7ffff94594d0, flags: CMSG_CLOEXEC ) = 200 220.788 (0.001 ms): dbus-daemon/10711 recvmsg(fd: 11, msg: 0x7ffff94594d0, flags: CMSG_CLOEXEC ) = -1 EAGAIN Resource temporarily unavailable 220.760 (0.004 ms): at-spi2-regist/10715 sendmsg(fd: 3, msg: 0x7fffdb8756c0, flags: NOSIGNAL ) = 200 220.771 (0.023 ms): perf/26347 open(filename: 0xf2e780, mode: 15918976 ) = 19 220.850 (0.002 ms): perf/26347 close(fd: 19 ) = 0 Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-6be5jvnkdzjptdrebfn5263n@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2013-09-12 12:35:21 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Use sched:sched_stat_runtime to provide a thread summary [root@sandy ~]# perf trace --sched --duration 0.100 --pid `pidof firefox` <SNIP> 17079.847 ( 0.009 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.892 ( 0.010 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable 17079.921 ( 0.013 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.949 ( 0.009 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ firefox - 17643 : 18013 [ 72.2% ] 359.110 ms firefox - 17663 : 41 [ 0.2% ] 21.439 ms firefox - 17664 : 6840 [ 27.4% ] 133.642 ms firefox - 17667 : 46 [ 0.2% ] 0.682 ms [root@sandy ~]# This is equivalent to the 'perf trace summary' subcomand in the tmp.perf/trace2 branch. Another example, setting a huge duration filter to get just a system wide summary: [root@sandy ~]# perf trace --duration 10000.0 --sched ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ scsi_eh_1 - 258 : 15 [ 0.0% ] 0.133 ms kworker/0:1H - 322 : 13 [ 0.0% ] 0.032 ms jbd2/dm-0-8 - 384 : 4 [ 0.0% ] 0.115 ms flush-253:0 - 470 : 1 [ 0.0% ] 0.027 ms firefox - 950 : 4783 [ 0.1% ] 24.863 ms firefox - 992 : 1883 [ 0.1% ] 6.808 ms firefox - 995 : 35 [ 0.0% ] 0.111 ms ksoftirqd/6 - 4362 : 2 [ 0.0% ] 0.005 ms ksoftirqd/7 - 4365 : 1 [ 0.0% ] 0.007 ms Xorg - 4671 : 148 [ 0.0% ] 0.912 ms gnome-settings- - 4846 : 14 [ 0.0% ] 0.086 ms seahorse-daemon - 4847 : 14 [ 0.0% ] 0.092 ms gnome-panel - 4875 : 46 [ 0.0% ] 0.159 ms gnome-power-man - 4918 : 16 [ 0.0% ] 0.065 ms gvfs-afc-volume - 4992 : 77 [ 0.0% ] 0.136 ms gnome-screensav - 5114 : 24 [ 0.0% ] 0.128 ms xchat - 8082 : 466 [ 0.0% ] 2.019 ms synergyc - 8369 : 941 [ 0.0% ] 3.291 ms synergyc - 8371 : 85 [ 0.0% ] 1.817 ms jbd2/dm-4-8 - 9352 : 4 [ 0.0% ] 0.109 ms rpcbind - 9786 : 3 [ 0.0% ] 0.017 ms rtkit-daemon - 12802 : 10 [ 0.0% ] 0.038 ms rtkit-daemon - 12803 : 8 [ 0.0% ] 0.000 ms udisks-daemon - 13020 : 27 [ 0.0% ] 0.240 ms kworker/7:0 - 14651 : 669 [ 0.0% ] 2.616 ms kworker/5:1 - 16220 : 2 [ 0.0% ] 0.069 ms kworker/4:0 - 19776 : 13 [ 0.0% ] 0.176 ms openvpn - 20131 : 133 [ 0.0% ] 0.762 ms plugin-containe - 20508 : 60658 [ 1.7% ] 131.153 ms npviewer.bin - 20520 : 72208 [ 2.0% ] 138.945 ms npviewer.bin - 20542 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 20543 : 30 [ 0.0% ] 0.074 ms npviewer.bin - 20547 : 35 [ 0.0% ] 0.092 ms npviewer.bin - 20552 : 35 [ 0.0% ] 0.093 ms sshd - 20645 : 32 [ 0.0% ] 0.071 ms npviewer.bin - 21053 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 21054 : 35 [ 0.0% ] 0.097 ms kworker/0:2 - 21169 : 149 [ 0.0% ] 1.143 ms kworker/3:0 - 22171 : 113 [ 0.0% ] 96.892 ms flush-253:4 - 22410 : 1 [ 0.0% ] 0.028 ms kworker/6:0 - 24581 : 25 [ 0.0% ] 0.275 ms kworker/1:0 - 25572 : 4 [ 0.0% ] 0.103 ms kworker/2:1 - 26299 : 138 [ 0.0% ] 1.440 ms kworker/0:0 - 26325 : 1 [ 0.0% ] 0.003 ms perf - 26330 : 3506967 [ 96.1% ] 6648.310 ms [root@sandy ~]# Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-mzuli0srnxyi1o029py6537x@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-17 17:13:12 -03:00
perf trace: Add possibility to switch off syscall events Currently, we may either trace syscalls or syscalls+pagefaults. We'd like to be able to trace *only* pagefaults and this commit implements this feature. Example: [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace --no-syscalls -F -p `pidof xchat` 0.000 ( 0.000 ms): xchat/4574 majfault [g_unichar_get_script+0x11] => /usr/lib64/libglib-2.0.so.0.3800.2@0xc403b (x.) 0.202 ( 0.000 ms): xchat/4574 majfault [_cairo_hash_table_lookup+0x53] => 0x2280ff0 (?.) 20.854 ( 0.000 ms): xchat/4574 majfault [gdk_cairo_set_source_pixbuf+0x110] => /usr/bin/xchat@0x6da1f (x.) 1022.000 ( 0.000 ms): xchat/4574 majfault [__memcpy_sse2_unaligned+0x29] => 0x7ff5a8ca0400 (?.) ^C[root@zoo /]# Below we can see malloc calls, 'trace' reading symbol tables in libraries to resolve symbols, etc. [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace --no-syscalls -F all --cpu 1 sleep 10 0.000 ( 0.000 ms): chrome/26589 minfault [0x1b53129] => /tmp/perf-26589.map@0x33cbcbf7f000 (x.) 96.477 ( 0.000 ms): libvirtd/947 minfault [copy_user_enhanced_fast_string+0x5] => 0x7f7685bba000 (?k) 113.164 ( 0.000 ms): Xorg/1063 minfault [0x786da] => 0x7fce52882a3c (?.) 7162.801 ( 0.000 ms): chrome/3747 minfault [0x8e1a89] => 0xfcaefed0008 (?.) <SNIP> 7773.138 ( 0.000 ms): chrome/3886 minfault [0x8e1a89] => 0xfcb0ce28008 (?.) 7992.022 ( 0.000 ms): chrome/26574 minfault [0x1b5a708] => 0x3de7b5fc5000 (?.) 8108.949 ( 0.000 ms): qemu-system-x8/4537 majfault [_int_malloc+0xee] => 0x7faffc466d60 (?.) 8108.975 ( 0.000 ms): qemu-system-x8/4537 minfault [_int_malloc+0x102] => 0x7faffc466d60 (?.) <SNIP> 8148.174 ( 0.000 ms): qemu-system-x8/4537 minfault [_int_malloc+0x102] => 0x7faffc4eb500 (?.) 8270.855 ( 0.000 ms): chrome/26245 minfault [do_bo_emit_reloc+0xdb] => 0x45d092bc004 (?.) 8270.869 ( 0.000 ms): chrome/26245 minfault [do_bo_emit_reloc+0x108] => 0x45d09150000 (?.) no symbols found in /usr/lib64/libspice-server.so.1.9.0, maybe install a debug package? 8273.831 ( 0.000 ms): trace/20198 majfault [__memcmp_sse4_1+0xbc6] => /usr/lib64/libspice-server.so.1.9.0@0xdf000 (d.) <SNIP> 8275.121 ( 0.000 ms): trace/20198 minfault [dso__load+0x38] => 0x14fe756 (?.) no symbols found in /usr/lib64/libelf-0.158.so, maybe install a debug package? 8275.142 ( 0.000 ms): trace/20198 minfault [__memcmp_sse4_1+0xbc6] => /usr/lib64/libelf-0.158.so@0x0 (d.) <SNIP> [root@zoo /]# Signed-off-by: Stanislav Fomichev <stfomichev@yandex-team.ru> Cc: David Ahern <dsahern@gmail.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1403799268-1367-6-git-send-email-stfomichev@yandex-team.ru Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-06-26 20:14:28 +04:00
perf trace: Support using -f to override perf.data file ownership Enable perf trace to use perf.data when it is not owned by current user or root. Example: # perf trace record ls # chown Yunlong.Song:Yunlong.Song perf.data # ls -al perf.data -rw------- 1 Yunlong.Song Yunlong.Song 4153101 Apr 2 15:28 perf.data # id uid=0(root) gid=0(root) groups=0(root),64(pkcs11) Before this patch: # perf trace -i perf.data File perf.data not owned by current user or root (use -f to override) # perf trace -i perf.data -f Error: unknown switch `f' usage: perf trace [<options>] [<command>] or: perf trace [<options>] -- <command> [<options>] or: perf trace record [<options>] [<command>] or: perf trace record [<options>] -- <command> [<options>] --event <event> event selector. use 'perf list' to list available events --comm show the thread COMM next to its id --tool_stats show tool stats -e, --expr <expr> list of events to trace -o, --output <file> output file name -i, --input <file> Analyze events in file -p, --pid <pid> trace events on existing process id -t, --tid <tid> trace events on existing thread id --filter-pids <float> ... As shown above, the -f option does not work at all. After this patch: # perf trace -i perf.data File perf.data not owned by current user or root (use -f to override) # perf trace -i perf.data -f 0.056 ( 0.002 ms): ls/47325 brk( ... 0.108 ( 0.018 ms): ls/47325 mmap(len: 4096, prot: READ|WRITE, ... 0.145 ( 0.013 ms): ls/47325 access(filename: 0x7f31259a0eb0, ... 0.172 ( 0.008 ms): ls/47325 open(filename: 0x7fffeb9a0d00, ... 0.180 ( 0.004 ms): ls/47325 stat(filename: 0x7fffeb9a0d00, ... 0.185 ( 0.004 ms): ls/47325 open(filename: 0x7fffeb9a0d00, ... 0.189 ( 0.003 ms): ls/47325 stat(filename: 0x7fffeb9a0d00, ... 0.195 ( 0.004 ms): ls/47325 open(filename: 0x7fffeb9a0d00, ... 0.199 ( 0.002 ms): ls/47325 stat(filename: 0x7fffeb9a0d00, ... 0.205 ( 0.004 ms): ls/47325 open(filename: 0x7fffeb9a0d00, ... 0.211 ( 0.004 ms): ls/47325 stat(filename: 0x7fffeb9a0d00, ... 0.220 ( 0.007 ms): ls/47325 open(filename: 0x7f312599e8ff, ... ... ... As shown above, the -f option really works now. Signed-off-by: Yunlong Song <yunlong.song@huawei.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Wang Nan <wangnan0@huawei.com> Link: http://lkml.kernel.org/r/1427982439-27388-10-git-send-email-yunlong.song@huawei.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-04-02 21:47:18 +08:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Allow mixing with other events Basically adopting 'perf record' --event command line argument syntax: # trace -e \!mprotect,mmap,munmap,open,close,read,fstat,access,arch_prctl --event sched:*switch,sched:*exec,sched:*exit usleep 1 0.048 ( ): sched:sched_process_exec:filename=/bin/usleep pid=24732 old_pid=24732) 0.078 (0.002 ms): usleep/24732 brk( ) = 0x78f000 0.430 (0.002 ms): usleep/24732 brk( ) = 0x78f000 0.434 (0.003 ms): usleep/24732 brk(brk: 0x7b0000 ) = 0x7b0000 0.438 (0.001 ms): usleep/24732 brk( ) = 0x7b0000 0.460 (0.004 ms): usleep/24732 nanosleep(rqtp: 0x7ffff3696a40) ... 0.460 ( ): sched:sched_switch:prev_comm=usleep prev_pid=24732 prev_prio=120 prev_state=S ==> next_comm=swapper/1 next_pid=0 next_prio=120) 0.515 (0.058 ms): usleep/24732 ... [continued]: nanosleep()) = 0 0.520 (0.000 ms): usleep/24732 exit_group( 0.550 ( ): sched:sched_process_exit:comm=usleep pid=24732 prio=120) # Next steps, probably in this order: 1) Use ordered_events code, the logic in trace needs the events to be time ordered when needed, i.e. when multiple CPUs are involved. 2) Callchains! 3) Automatically account for interruptions when saying how long things took. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-gpst8mph575yb4wgf91qibyb@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-03 12:58:57 -03:00
perf trace: Allow mixing with other events Basically adopting 'perf record' --event command line argument syntax: # trace -e \!mprotect,mmap,munmap,open,close,read,fstat,access,arch_prctl --event sched:*switch,sched:*exec,sched:*exit usleep 1 0.048 ( ): sched:sched_process_exec:filename=/bin/usleep pid=24732 old_pid=24732) 0.078 (0.002 ms): usleep/24732 brk( ) = 0x78f000 0.430 (0.002 ms): usleep/24732 brk( ) = 0x78f000 0.434 (0.003 ms): usleep/24732 brk(brk: 0x7b0000 ) = 0x7b0000 0.438 (0.001 ms): usleep/24732 brk( ) = 0x7b0000 0.460 (0.004 ms): usleep/24732 nanosleep(rqtp: 0x7ffff3696a40) ... 0.460 ( ): sched:sched_switch:prev_comm=usleep prev_pid=24732 prev_prio=120 prev_state=S ==> next_comm=swapper/1 next_pid=0 next_prio=120) 0.515 (0.058 ms): usleep/24732 ... [continued]: nanosleep()) = 0 0.520 (0.000 ms): usleep/24732 exit_group( 0.550 ( ): sched:sched_process_exit:comm=usleep pid=24732 prio=120) # Next steps, probably in this order: 1) Use ordered_events code, the logic in trace needs the events to be time ordered when needed, i.e. when multiple CPUs are involved. 2) Callchains! 3) Automatically account for interruptions when saying how long things took. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-gpst8mph575yb4wgf91qibyb@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-03 12:58:57 -03:00
perf trace: Allow mixing with other events Basically adopting 'perf record' --event command line argument syntax: # trace -e \!mprotect,mmap,munmap,open,close,read,fstat,access,arch_prctl --event sched:*switch,sched:*exec,sched:*exit usleep 1 0.048 ( ): sched:sched_process_exec:filename=/bin/usleep pid=24732 old_pid=24732) 0.078 (0.002 ms): usleep/24732 brk( ) = 0x78f000 0.430 (0.002 ms): usleep/24732 brk( ) = 0x78f000 0.434 (0.003 ms): usleep/24732 brk(brk: 0x7b0000 ) = 0x7b0000 0.438 (0.001 ms): usleep/24732 brk( ) = 0x7b0000 0.460 (0.004 ms): usleep/24732 nanosleep(rqtp: 0x7ffff3696a40) ... 0.460 ( ): sched:sched_switch:prev_comm=usleep prev_pid=24732 prev_prio=120 prev_state=S ==> next_comm=swapper/1 next_pid=0 next_prio=120) 0.515 (0.058 ms): usleep/24732 ... [continued]: nanosleep()) = 0 0.520 (0.000 ms): usleep/24732 exit_group( 0.550 ( ): sched:sched_process_exit:comm=usleep pid=24732 prio=120) # Next steps, probably in this order: 1) Use ordered_events code, the logic in trace needs the events to be time ordered when needed, i.e. when multiple CPUs are involved. 2) Callchains! 3) Automatically account for interruptions when saying how long things took. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Don Zickus <dzickus@redhat.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-gpst8mph575yb4wgf91qibyb@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-02-03 12:58:57 -03:00
perf trace: Add pagefaults record and replay support Previous commit added live pagefault trace support, this one adds record and replay support. Example: [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace -F all record -a sleep 10 [ perf record: Woken up 0 times to write data ] [ perf record: Captured and wrote 1029.722 MB perf.data (~44989242 samples) ] [root@zoo /]# ls -la perf.data -rw-------. 1 root root 1083921722 Jun 26 17:44 perf.data [root@zoo /]# perf evlist raw_syscalls:sys_enter raw_syscalls:sys_exit major-faults minor-faults [root@zoo /]# trace -i perf.data | grep -v trace\/ | tail -15 156.137 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.139 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) 156.140 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0xc4] => 0x0 (?.) 156.144 ( 0.000 ms): perl/18476 minfault [_int_free+0xda] => 0x0 (?.) 156.151 ( 0.000 ms): perl/18476 minfault [_int_free+0x1df] => 0x0 (?.) 156.158 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.161 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.168 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.172 ( 0.000 ms): perl/18476 minfault [0xb4243] => 0x0 (?.) 156.173 ( 0.000 ms): perl/18476 minfault [_int_free+0xda] => 0x0 (?.) 156.183 ( 0.000 ms): perl/18476 minfault [Perl_hfree_next_entry+0xb4] => 0x0 (?.) 156.197 ( 0.000 ms): perl/18476 minfault [_int_free+0x1df] => 0x0 (?.) 156.216 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) 156.221 ( 0.000 ms): perl/18476 minfault [Perl_sv_clear+0x123] => 0x0 (?.) [root@zoo /]# Signed-off-by: Stanislav Fomichev <stfomichev@yandex-team.ru> Cc: David Ahern <dsahern@gmail.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1403799268-1367-4-git-send-email-stfomichev@yandex-team.ru Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-06-26 20:14:26 +04:00
perf trace: Add possibility to switch off syscall events Currently, we may either trace syscalls or syscalls+pagefaults. We'd like to be able to trace *only* pagefaults and this commit implements this feature. Example: [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace --no-syscalls -F -p `pidof xchat` 0.000 ( 0.000 ms): xchat/4574 majfault [g_unichar_get_script+0x11] => /usr/lib64/libglib-2.0.so.0.3800.2@0xc403b (x.) 0.202 ( 0.000 ms): xchat/4574 majfault [_cairo_hash_table_lookup+0x53] => 0x2280ff0 (?.) 20.854 ( 0.000 ms): xchat/4574 majfault [gdk_cairo_set_source_pixbuf+0x110] => /usr/bin/xchat@0x6da1f (x.) 1022.000 ( 0.000 ms): xchat/4574 majfault [__memcpy_sse2_unaligned+0x29] => 0x7ff5a8ca0400 (?.) ^C[root@zoo /]# Below we can see malloc calls, 'trace' reading symbol tables in libraries to resolve symbols, etc. [root@zoo /]# echo 1 > /proc/sys/vm/drop_caches ; trace --no-syscalls -F all --cpu 1 sleep 10 0.000 ( 0.000 ms): chrome/26589 minfault [0x1b53129] => /tmp/perf-26589.map@0x33cbcbf7f000 (x.) 96.477 ( 0.000 ms): libvirtd/947 minfault [copy_user_enhanced_fast_string+0x5] => 0x7f7685bba000 (?k) 113.164 ( 0.000 ms): Xorg/1063 minfault [0x786da] => 0x7fce52882a3c (?.) 7162.801 ( 0.000 ms): chrome/3747 minfault [0x8e1a89] => 0xfcaefed0008 (?.) <SNIP> 7773.138 ( 0.000 ms): chrome/3886 minfault [0x8e1a89] => 0xfcb0ce28008 (?.) 7992.022 ( 0.000 ms): chrome/26574 minfault [0x1b5a708] => 0x3de7b5fc5000 (?.) 8108.949 ( 0.000 ms): qemu-system-x8/4537 majfault [_int_malloc+0xee] => 0x7faffc466d60 (?.) 8108.975 ( 0.000 ms): qemu-system-x8/4537 minfault [_int_malloc+0x102] => 0x7faffc466d60 (?.) <SNIP> 8148.174 ( 0.000 ms): qemu-system-x8/4537 minfault [_int_malloc+0x102] => 0x7faffc4eb500 (?.) 8270.855 ( 0.000 ms): chrome/26245 minfault [do_bo_emit_reloc+0xdb] => 0x45d092bc004 (?.) 8270.869 ( 0.000 ms): chrome/26245 minfault [do_bo_emit_reloc+0x108] => 0x45d09150000 (?.) no symbols found in /usr/lib64/libspice-server.so.1.9.0, maybe install a debug package? 8273.831 ( 0.000 ms): trace/20198 majfault [__memcmp_sse4_1+0xbc6] => /usr/lib64/libspice-server.so.1.9.0@0xdf000 (d.) <SNIP> 8275.121 ( 0.000 ms): trace/20198 minfault [dso__load+0x38] => 0x14fe756 (?.) no symbols found in /usr/lib64/libelf-0.158.so, maybe install a debug package? 8275.142 ( 0.000 ms): trace/20198 minfault [__memcmp_sse4_1+0xbc6] => /usr/lib64/libelf-0.158.so@0x0 (d.) <SNIP> [root@zoo /]# Signed-off-by: Stanislav Fomichev <stfomichev@yandex-team.ru> Cc: David Ahern <dsahern@gmail.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Link: http://lkml.kernel.org/r/1403799268-1367-6-git-send-email-stfomichev@yandex-team.ru Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-06-26 20:14:28 +04:00
perf trace: Support 'strace' syscall event groups I.e.: $ cat ~/share/perf-core/strace/groups/file access chmod creat execve faccessat getcwd lstat mkdir open openat quotactl readlink rename rmdir stat statfs symlink unlink $ Then, on a quiet desktop, try running this and then moving your mouse to see the deluge of mouse related activity: # perf probe 'vfs_getname=getname_flags:72 pathname=filename:string' Added new event: probe:vfs_getname (on getname_flags:72 with pathname=filename:string) You can now use it in all perf tools, such as: perf record -e probe:vfs_getname -aR sleep 1 # # trace --ev probe:vfs_getname --filter-pids 2232 -e file 0.042 (0.042 ms): mousetweaks/2235 open(filename: 0x14e3910, mode: 438 ) ... 0.042 ( ): probe:vfs_getname:(ffffffff812230bc) pathname="/home/acme/.icons/Adwaita/cursors/xterm") 0.100 (0.100 ms): mousetweaks/2235 ... [continued]: open()) = -1 ENOENT No such file or directory 0.142 (0.018 ms): mousetweaks/2235 open(filename: 0x14c3c10, mode: 438 ) ... 0.142 ( ): probe:vfs_getname:(ffffffff812230bc) pathname="/home/acme/.icons/Adwaita/index.theme") 0.192 (0.069 ms): mousetweaks/2235 ... [continued]: open()) = -1 ENOENT No such file or directory 0.230 (0.017 ms): mousetweaks/2235 open(filename: 0x14c3c10, mode: 438 ) ... 0.230 ( ): probe:vfs_getname:(ffffffff812230bc) pathname="/usr/share/icons/Adwaita/cursors/xterm") 0.253 (0.041 ms): mousetweaks/2235 ... [continued]: open()) = 14 0.459 (0.008 ms): mousetweaks/2235 open(filename: 0x14e3910, mode: 438 ) ... 0.459 ( ): probe:vfs_getname:(ffffffff812230bc) pathname="/home/acme/.icons/Adwaita/cursors/left_side") 0.468 (0.017 ms): mousetweaks/2235 ... [continued]: open()) = -1 ENOENT No such file or directory Need to combine that raw_syscalls:sys_enter(open) + probe:vfs_getname + raw_syscalls:sys_exit(open) sequence... Now, if you're bored, please write some more syscall groups, like the ones in 'strace' and send it our way :-) Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Milian Wolff <mail@milianw.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-a42xklu59lcbxp7bbnic74a8@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-07-20 12:02:09 -03:00
perf trace: Support 'strace' syscall event groups I.e.: $ cat ~/share/perf-core/strace/groups/file access chmod creat execve faccessat getcwd lstat mkdir open openat quotactl readlink rename rmdir stat statfs symlink unlink $ Then, on a quiet desktop, try running this and then moving your mouse to see the deluge of mouse related activity: # perf probe 'vfs_getname=getname_flags:72 pathname=filename:string' Added new event: probe:vfs_getname (on getname_flags:72 with pathname=filename:string) You can now use it in all perf tools, such as: perf record -e probe:vfs_getname -aR sleep 1 # # trace --ev probe:vfs_getname --filter-pids 2232 -e file 0.042 (0.042 ms): mousetweaks/2235 open(filename: 0x14e3910, mode: 438 ) ... 0.042 ( ): probe:vfs_getname:(ffffffff812230bc) pathname="/home/acme/.icons/Adwaita/cursors/xterm") 0.100 (0.100 ms): mousetweaks/2235 ... [continued]: open()) = -1 ENOENT No such file or directory 0.142 (0.018 ms): mousetweaks/2235 open(filename: 0x14c3c10, mode: 438 ) ... 0.142 ( ): probe:vfs_getname:(ffffffff812230bc) pathname="/home/acme/.icons/Adwaita/index.theme") 0.192 (0.069 ms): mousetweaks/2235 ... [continued]: open()) = -1 ENOENT No such file or directory 0.230 (0.017 ms): mousetweaks/2235 open(filename: 0x14c3c10, mode: 438 ) ... 0.230 ( ): probe:vfs_getname:(ffffffff812230bc) pathname="/usr/share/icons/Adwaita/cursors/xterm") 0.253 (0.041 ms): mousetweaks/2235 ... [continued]: open()) = 14 0.459 (0.008 ms): mousetweaks/2235 open(filename: 0x14e3910, mode: 438 ) ... 0.459 ( ): probe:vfs_getname:(ffffffff812230bc) pathname="/home/acme/.icons/Adwaita/cursors/left_side") 0.468 (0.017 ms): mousetweaks/2235 ... [continued]: open()) = -1 ENOENT No such file or directory Need to combine that raw_syscalls:sys_enter(open) + probe:vfs_getname + raw_syscalls:sys_exit(open) sequence... Now, if you're bored, please write some more syscall groups, like the ones in 'strace' and send it our way :-) Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Borislav Petkov <bp@suse.de> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Milian Wolff <mail@milianw.de> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/n/tip-a42xklu59lcbxp7bbnic74a8@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-07-20 12:02:09 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00
perf trace: Use sched:sched_stat_runtime to provide a thread summary [root@sandy ~]# perf trace --sched --duration 0.100 --pid `pidof firefox` <SNIP> 17079.847 ( 0.009 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.892 ( 0.010 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable 17079.921 ( 0.013 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.949 ( 0.009 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ firefox - 17643 : 18013 [ 72.2% ] 359.110 ms firefox - 17663 : 41 [ 0.2% ] 21.439 ms firefox - 17664 : 6840 [ 27.4% ] 133.642 ms firefox - 17667 : 46 [ 0.2% ] 0.682 ms [root@sandy ~]# This is equivalent to the 'perf trace summary' subcomand in the tmp.perf/trace2 branch. Another example, setting a huge duration filter to get just a system wide summary: [root@sandy ~]# perf trace --duration 10000.0 --sched ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ scsi_eh_1 - 258 : 15 [ 0.0% ] 0.133 ms kworker/0:1H - 322 : 13 [ 0.0% ] 0.032 ms jbd2/dm-0-8 - 384 : 4 [ 0.0% ] 0.115 ms flush-253:0 - 470 : 1 [ 0.0% ] 0.027 ms firefox - 950 : 4783 [ 0.1% ] 24.863 ms firefox - 992 : 1883 [ 0.1% ] 6.808 ms firefox - 995 : 35 [ 0.0% ] 0.111 ms ksoftirqd/6 - 4362 : 2 [ 0.0% ] 0.005 ms ksoftirqd/7 - 4365 : 1 [ 0.0% ] 0.007 ms Xorg - 4671 : 148 [ 0.0% ] 0.912 ms gnome-settings- - 4846 : 14 [ 0.0% ] 0.086 ms seahorse-daemon - 4847 : 14 [ 0.0% ] 0.092 ms gnome-panel - 4875 : 46 [ 0.0% ] 0.159 ms gnome-power-man - 4918 : 16 [ 0.0% ] 0.065 ms gvfs-afc-volume - 4992 : 77 [ 0.0% ] 0.136 ms gnome-screensav - 5114 : 24 [ 0.0% ] 0.128 ms xchat - 8082 : 466 [ 0.0% ] 2.019 ms synergyc - 8369 : 941 [ 0.0% ] 3.291 ms synergyc - 8371 : 85 [ 0.0% ] 1.817 ms jbd2/dm-4-8 - 9352 : 4 [ 0.0% ] 0.109 ms rpcbind - 9786 : 3 [ 0.0% ] 0.017 ms rtkit-daemon - 12802 : 10 [ 0.0% ] 0.038 ms rtkit-daemon - 12803 : 8 [ 0.0% ] 0.000 ms udisks-daemon - 13020 : 27 [ 0.0% ] 0.240 ms kworker/7:0 - 14651 : 669 [ 0.0% ] 2.616 ms kworker/5:1 - 16220 : 2 [ 0.0% ] 0.069 ms kworker/4:0 - 19776 : 13 [ 0.0% ] 0.176 ms openvpn - 20131 : 133 [ 0.0% ] 0.762 ms plugin-containe - 20508 : 60658 [ 1.7% ] 131.153 ms npviewer.bin - 20520 : 72208 [ 2.0% ] 138.945 ms npviewer.bin - 20542 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 20543 : 30 [ 0.0% ] 0.074 ms npviewer.bin - 20547 : 35 [ 0.0% ] 0.092 ms npviewer.bin - 20552 : 35 [ 0.0% ] 0.093 ms sshd - 20645 : 32 [ 0.0% ] 0.071 ms npviewer.bin - 21053 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 21054 : 35 [ 0.0% ] 0.097 ms kworker/0:2 - 21169 : 149 [ 0.0% ] 1.143 ms kworker/3:0 - 22171 : 113 [ 0.0% ] 96.892 ms flush-253:4 - 22410 : 1 [ 0.0% ] 0.028 ms kworker/6:0 - 24581 : 25 [ 0.0% ] 0.275 ms kworker/1:0 - 25572 : 4 [ 0.0% ] 0.103 ms kworker/2:1 - 26299 : 138 [ 0.0% ] 1.440 ms kworker/0:0 - 26325 : 1 [ 0.0% ] 0.003 ms perf - 26330 : 3506967 [ 96.1% ] 6648.310 ms [root@sandy ~]# Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-mzuli0srnxyi1o029py6537x@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-17 17:13:12 -03:00
perf trace: Use sched:sched_stat_runtime to provide a thread summary [root@sandy ~]# perf trace --sched --duration 0.100 --pid `pidof firefox` <SNIP> 17079.847 ( 0.009 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.892 ( 0.010 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable 17079.921 ( 0.013 ms): 17643 poll(ufds: 140037623086496, nfds: 11, timeout_msecs: 0) = 0 Timeout 17079.949 ( 0.009 ms): 17643 read(fd: 4, buf: 140038178943092, count: 4096 ) = -1 EAGAIN Resource temporarily unavailable ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ firefox - 17643 : 18013 [ 72.2% ] 359.110 ms firefox - 17663 : 41 [ 0.2% ] 21.439 ms firefox - 17664 : 6840 [ 27.4% ] 133.642 ms firefox - 17667 : 46 [ 0.2% ] 0.682 ms [root@sandy ~]# This is equivalent to the 'perf trace summary' subcomand in the tmp.perf/trace2 branch. Another example, setting a huge duration filter to get just a system wide summary: [root@sandy ~]# perf trace --duration 10000.0 --sched ^C _____________________________________________________________________ __) Summary of events (__ [ task - pid ] [ events ] [ ratio ] [ runtime ] _____________________________________________________________________ scsi_eh_1 - 258 : 15 [ 0.0% ] 0.133 ms kworker/0:1H - 322 : 13 [ 0.0% ] 0.032 ms jbd2/dm-0-8 - 384 : 4 [ 0.0% ] 0.115 ms flush-253:0 - 470 : 1 [ 0.0% ] 0.027 ms firefox - 950 : 4783 [ 0.1% ] 24.863 ms firefox - 992 : 1883 [ 0.1% ] 6.808 ms firefox - 995 : 35 [ 0.0% ] 0.111 ms ksoftirqd/6 - 4362 : 2 [ 0.0% ] 0.005 ms ksoftirqd/7 - 4365 : 1 [ 0.0% ] 0.007 ms Xorg - 4671 : 148 [ 0.0% ] 0.912 ms gnome-settings- - 4846 : 14 [ 0.0% ] 0.086 ms seahorse-daemon - 4847 : 14 [ 0.0% ] 0.092 ms gnome-panel - 4875 : 46 [ 0.0% ] 0.159 ms gnome-power-man - 4918 : 16 [ 0.0% ] 0.065 ms gvfs-afc-volume - 4992 : 77 [ 0.0% ] 0.136 ms gnome-screensav - 5114 : 24 [ 0.0% ] 0.128 ms xchat - 8082 : 466 [ 0.0% ] 2.019 ms synergyc - 8369 : 941 [ 0.0% ] 3.291 ms synergyc - 8371 : 85 [ 0.0% ] 1.817 ms jbd2/dm-4-8 - 9352 : 4 [ 0.0% ] 0.109 ms rpcbind - 9786 : 3 [ 0.0% ] 0.017 ms rtkit-daemon - 12802 : 10 [ 0.0% ] 0.038 ms rtkit-daemon - 12803 : 8 [ 0.0% ] 0.000 ms udisks-daemon - 13020 : 27 [ 0.0% ] 0.240 ms kworker/7:0 - 14651 : 669 [ 0.0% ] 2.616 ms kworker/5:1 - 16220 : 2 [ 0.0% ] 0.069 ms kworker/4:0 - 19776 : 13 [ 0.0% ] 0.176 ms openvpn - 20131 : 133 [ 0.0% ] 0.762 ms plugin-containe - 20508 : 60658 [ 1.7% ] 131.153 ms npviewer.bin - 20520 : 72208 [ 2.0% ] 138.945 ms npviewer.bin - 20542 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 20543 : 30 [ 0.0% ] 0.074 ms npviewer.bin - 20547 : 35 [ 0.0% ] 0.092 ms npviewer.bin - 20552 : 35 [ 0.0% ] 0.093 ms sshd - 20645 : 32 [ 0.0% ] 0.071 ms npviewer.bin - 21053 : 35 [ 0.0% ] 0.074 ms npviewer.bin - 21054 : 35 [ 0.0% ] 0.097 ms kworker/0:2 - 21169 : 149 [ 0.0% ] 1.143 ms kworker/3:0 - 22171 : 113 [ 0.0% ] 96.892 ms flush-253:4 - 22410 : 1 [ 0.0% ] 0.028 ms kworker/6:0 - 24581 : 25 [ 0.0% ] 0.275 ms kworker/1:0 - 25572 : 4 [ 0.0% ] 0.103 ms kworker/2:1 - 26299 : 138 [ 0.0% ] 1.440 ms kworker/0:0 - 26325 : 1 [ 0.0% ] 0.003 ms perf - 26330 : 3506967 [ 96.1% ] 6648.310 ms [root@sandy ~]# Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-mzuli0srnxyi1o029py6537x@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-10-17 17:13:12 -03:00
perf trace: New tool Initially should look loosely like the venerable 'strace' tool, but using the infrastructure in the perf tools to allow tracing extra targets: [acme@sandy linux]$ perf trace --hell Error: unknown option `hell' usage: perf trace <PID> -p, --pid <pid> trace events on existing process id --tid <tid> trace events on existing thread id --all-cpus system-wide collection from all CPUs --cpu <cpu> list of cpus to monitor --no-inherit child tasks do not inherit counters --mmap-pages <n> number of mmap data pages --uid <user> user to profile [acme@sandy linux]$ Those should have the same semantics as when using with 'perf record'. It gets stuck sometimes, but hey, it works sometimes too! In time it should support perf.data based workloads, i.e. it should have a: -o filename Command line option that will produce a perf.data file that can then be used with 'perf trace' or any of the other perf tools (script, report, etc). It will also eventually have the set of functionalities described in the previous 'trace' prototype by Thomas Gleixner: "Announcing a new utility: 'trace'" http://lwn.net/Articles/415728/ Also planned is to have some of the features suggested in the comments of that LWN article. Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Mike Galbraith <efault@gmx.de> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Thomas Gleixner <tglx@linutronix.de> Link: http://lkml.kernel.org/n/tip-v9x3q9rv4caxtox7wtjpchq5@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2012-09-26 20:05:56 -03:00