linux/tools/perf/scripts/python/export-to-sqlite.py

799 lines
24 KiB

perf scripts python: export-to-sqlite.py: Export IPC information Export cycle and instruction counts on samples and calls tables. Committer testing: First runs some workload collecting intel_pt with the 'cyc' ter just for userspace: [root@quaco adrian.hunter]# perf record -o simple-retpoline.perf.data -e intel_pt/cyc/u ./simple-retpoline [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.035 MB simple-retpoline.perf.data ] [root@quaco adrian.hunter]# Then use the export-to-sqlite.py script to see if the changes in this cset don't make it to break and if the changes in the db schema are the ones expected: [root@quaco adrian.hunter]# perf script -i simple-retpoline.perf.data --itrace=be -s ~acme/libexec/perf-core/scripts/python/export-to-sqlite.py simple-retpoline.db branches calls 2019-05-31 11:50:46.942710 Creating database ... 2019-05-31 11:50:46.949663 Writing records... 2019-05-31 11:50:47.224033 Adding indexes 2019-05-31 11:50:47.231599 Done [root@quaco adrian.hunter]# Now lets use the db: [root@quaco adrian.hunter]# sqlite3 simple-retpoline.db SQLite version 3.26.0 2018-12-01 12:34:55 Enter ".help" for usage hints. sqlite> .schema samples CREATE TABLE samples (id integer NOT NULL PRIMARY KEY,evsel_id bigint,machine_id bigint,thread_id bigint,comm_id bigint,dso_id bigint,symbol_id bigint,sym_offset bigint,ip bigint,time bigint,cpuinteger,to_dso_id bigint,to_symbol_id bigint,to_sym_offset bigint,to_ip bigint,branch_type integer,in_tx boolean,call_path_id bigint,insn_count bigint,cyc_count bigint); sqlite> Cool, the 'insn_count' and 'cyc_count' are there, now lets see if we can use them in a query: sqlite> select insn_count,cyc_count from samples where cyc_count > 1500 and insn_count < 10; 6|1507 sqlite> select insn_count,cyc_count from samples where cyc_count > 1500; 118|2210 140|1516 3783|1861 132|1521 6|1507 sqlite> Seems to work :-) Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190520113728.14389-17-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-05-20 14:37:22 +03:00
perf scripts python: export-to-sqlite.py: Export IPC information Export cycle and instruction counts on samples and calls tables. Committer testing: First runs some workload collecting intel_pt with the 'cyc' ter just for userspace: [root@quaco adrian.hunter]# perf record -o simple-retpoline.perf.data -e intel_pt/cyc/u ./simple-retpoline [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.035 MB simple-retpoline.perf.data ] [root@quaco adrian.hunter]# Then use the export-to-sqlite.py script to see if the changes in this cset don't make it to break and if the changes in the db schema are the ones expected: [root@quaco adrian.hunter]# perf script -i simple-retpoline.perf.data --itrace=be -s ~acme/libexec/perf-core/scripts/python/export-to-sqlite.py simple-retpoline.db branches calls 2019-05-31 11:50:46.942710 Creating database ... 2019-05-31 11:50:46.949663 Writing records... 2019-05-31 11:50:47.224033 Adding indexes 2019-05-31 11:50:47.231599 Done [root@quaco adrian.hunter]# Now lets use the db: [root@quaco adrian.hunter]# sqlite3 simple-retpoline.db SQLite version 3.26.0 2018-12-01 12:34:55 Enter ".help" for usage hints. sqlite> .schema samples CREATE TABLE samples (id integer NOT NULL PRIMARY KEY,evsel_id bigint,machine_id bigint,thread_id bigint,comm_id bigint,dso_id bigint,symbol_id bigint,sym_offset bigint,ip bigint,time bigint,cpuinteger,to_dso_id bigint,to_symbol_id bigint,to_sym_offset bigint,to_ip bigint,branch_type integer,in_tx boolean,call_path_id bigint,insn_count bigint,cyc_count bigint); sqlite> Cool, the 'insn_count' and 'cyc_count' are there, now lets see if we can use them in a query: sqlite> select insn_count,cyc_count from samples where cyc_count > 1500 and insn_count < 10; 6|1507 sqlite> select insn_count,cyc_count from samples where cyc_count > 1500; 118|2210 140|1516 3783|1861 132|1521 6|1507 sqlite> Seems to work :-) Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190520113728.14389-17-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-05-20 14:37:22 +03:00
perf scripts python: export-to-sqlite.py: Export IPC information Export cycle and instruction counts on samples and calls tables. Committer testing: First runs some workload collecting intel_pt with the 'cyc' ter just for userspace: [root@quaco adrian.hunter]# perf record -o simple-retpoline.perf.data -e intel_pt/cyc/u ./simple-retpoline [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.035 MB simple-retpoline.perf.data ] [root@quaco adrian.hunter]# Then use the export-to-sqlite.py script to see if the changes in this cset don't make it to break and if the changes in the db schema are the ones expected: [root@quaco adrian.hunter]# perf script -i simple-retpoline.perf.data --itrace=be -s ~acme/libexec/perf-core/scripts/python/export-to-sqlite.py simple-retpoline.db branches calls 2019-05-31 11:50:46.942710 Creating database ... 2019-05-31 11:50:46.949663 Writing records... 2019-05-31 11:50:47.224033 Adding indexes 2019-05-31 11:50:47.231599 Done [root@quaco adrian.hunter]# Now lets use the db: [root@quaco adrian.hunter]# sqlite3 simple-retpoline.db SQLite version 3.26.0 2018-12-01 12:34:55 Enter ".help" for usage hints. sqlite> .schema samples CREATE TABLE samples (id integer NOT NULL PRIMARY KEY,evsel_id bigint,machine_id bigint,thread_id bigint,comm_id bigint,dso_id bigint,symbol_id bigint,sym_offset bigint,ip bigint,time bigint,cpuinteger,to_dso_id bigint,to_symbol_id bigint,to_sym_offset bigint,to_ip bigint,branch_type integer,in_tx boolean,call_path_id bigint,insn_count bigint,cyc_count bigint); sqlite> Cool, the 'insn_count' and 'cyc_count' are there, now lets see if we can use them in a query: sqlite> select insn_count,cyc_count from samples where cyc_count > 1500 and insn_count < 10; 6|1507 sqlite> select insn_count,cyc_count from samples where cyc_count > 1500; 118|2210 140|1516 3783|1861 132|1521 6|1507 sqlite> Seems to work :-) Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190520113728.14389-17-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-05-20 14:37:22 +03:00
perf scripts python: export-to-sqlite.py: Export IPC information Export cycle and instruction counts on samples and calls tables. Committer testing: First runs some workload collecting intel_pt with the 'cyc' ter just for userspace: [root@quaco adrian.hunter]# perf record -o simple-retpoline.perf.data -e intel_pt/cyc/u ./simple-retpoline [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.035 MB simple-retpoline.perf.data ] [root@quaco adrian.hunter]# Then use the export-to-sqlite.py script to see if the changes in this cset don't make it to break and if the changes in the db schema are the ones expected: [root@quaco adrian.hunter]# perf script -i simple-retpoline.perf.data --itrace=be -s ~acme/libexec/perf-core/scripts/python/export-to-sqlite.py simple-retpoline.db branches calls 2019-05-31 11:50:46.942710 Creating database ... 2019-05-31 11:50:46.949663 Writing records... 2019-05-31 11:50:47.224033 Adding indexes 2019-05-31 11:50:47.231599 Done [root@quaco adrian.hunter]# Now lets use the db: [root@quaco adrian.hunter]# sqlite3 simple-retpoline.db SQLite version 3.26.0 2018-12-01 12:34:55 Enter ".help" for usage hints. sqlite> .schema samples CREATE TABLE samples (id integer NOT NULL PRIMARY KEY,evsel_id bigint,machine_id bigint,thread_id bigint,comm_id bigint,dso_id bigint,symbol_id bigint,sym_offset bigint,ip bigint,time bigint,cpuinteger,to_dso_id bigint,to_symbol_id bigint,to_sym_offset bigint,to_ip bigint,branch_type integer,in_tx boolean,call_path_id bigint,insn_count bigint,cyc_count bigint); sqlite> Cool, the 'insn_count' and 'cyc_count' are there, now lets see if we can use them in a query: sqlite> select insn_count,cyc_count from samples where cyc_count > 1500 and insn_count < 10; 6|1507 sqlite> select insn_count,cyc_count from samples where cyc_count > 1500; 118|2210 140|1516 3783|1861 132|1521 6|1507 sqlite> Seems to work :-) Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190520113728.14389-17-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-05-20 14:37:22 +03:00
perf thread-stack: Represent jmps to the start of a different symbol The compiler might optimize a call/ret combination by making it a jmp. However the thread-stack does not presently cater for that, so that such control flow is not visible in the call graph. Make it visible by recording on the stack a branch to the start of a different symbol. Note, that means when a ret pops the stack, all jmps must be popped off first. Example: $ cat jmp-to-fn.c __attribute__((noinline)) int bar(void) { return -1; } __attribute__((noinline)) int foo(void) { return bar() + 1; } int main() { return foo(); } $ gcc -ggdb3 -Wall -Wextra -O2 -o jmp-to-fn jmp-to-fn.c $ objdump -d jmp-to-fn <SNIP> 0000000000001040 <main>: 1040: 31 c0 xor %eax,%eax 1042: e9 09 01 00 00 jmpq 1150 <foo> <SNIP> 0000000000001140 <bar>: 1140: b8 ff ff ff ff mov $0xffffffff,%eax 1145: c3 retq <SNIP> 0000000000001150 <foo>: 1150: 31 c0 xor %eax,%eax 1152: e8 e9 ff ff ff callq 1140 <bar> 1157: 83 c0 01 add $0x1,%eax 115a: c3 retq <SNIP> $ perf record -o jmp-to-fn.perf.data -e intel_pt/cyc/u ./jmp-to-fn [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0,017 MB jmp-to-fn.perf.data ] $ perf script -i jmp-to-fn.perf.data --itrace=be -s ~/libexec/perf-core/scripts/python/export-to-sqlite.py jmp-to-fn.db branches calls 2019-01-08 13:24:58.783069 Creating database... 2019-01-08 13:24:58.794650 Writing records... 2019-01-08 13:24:59.008050 Adding indexes 2019-01-08 13:24:59.015802 Done $ ~/libexec/perf-core/scripts/python/exported-sql-viewer.py jmp-to-fn.db Before: main -> bar After: main -> foo -> bar Committer testing: Install the python2-pyside package, then select these menu options on the GUI: "Reports" "Context sensitive callgraphs" Then go on expanding the symbols, to get, full picture when doing this on a fedora:29 with gcc version 8.2.1 20181215 (Red Hat 8.2.1-6) (GCC): jmp-to-fn PID:TID _start (ld-2.28.so) __libc_start_main main foo bar To verify that indeed, this fixes the problem. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Acked-by: Jiri Olsa <jolsa@kernel.org> Link: http://lkml.kernel.org/r/20190109091835.5570-5-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-01-09 11:18:33 +02:00
perf scripts python: export-to-sqlite.py: Export IPC information Export cycle and instruction counts on samples and calls tables. Committer testing: First runs some workload collecting intel_pt with the 'cyc' ter just for userspace: [root@quaco adrian.hunter]# perf record -o simple-retpoline.perf.data -e intel_pt/cyc/u ./simple-retpoline [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.035 MB simple-retpoline.perf.data ] [root@quaco adrian.hunter]# Then use the export-to-sqlite.py script to see if the changes in this cset don't make it to break and if the changes in the db schema are the ones expected: [root@quaco adrian.hunter]# perf script -i simple-retpoline.perf.data --itrace=be -s ~acme/libexec/perf-core/scripts/python/export-to-sqlite.py simple-retpoline.db branches calls 2019-05-31 11:50:46.942710 Creating database ... 2019-05-31 11:50:46.949663 Writing records... 2019-05-31 11:50:47.224033 Adding indexes 2019-05-31 11:50:47.231599 Done [root@quaco adrian.hunter]# Now lets use the db: [root@quaco adrian.hunter]# sqlite3 simple-retpoline.db SQLite version 3.26.0 2018-12-01 12:34:55 Enter ".help" for usage hints. sqlite> .schema samples CREATE TABLE samples (id integer NOT NULL PRIMARY KEY,evsel_id bigint,machine_id bigint,thread_id bigint,comm_id bigint,dso_id bigint,symbol_id bigint,sym_offset bigint,ip bigint,time bigint,cpuinteger,to_dso_id bigint,to_symbol_id bigint,to_sym_offset bigint,to_ip bigint,branch_type integer,in_tx boolean,call_path_id bigint,insn_count bigint,cyc_count bigint); sqlite> Cool, the 'insn_count' and 'cyc_count' are there, now lets see if we can use them in a query: sqlite> select insn_count,cyc_count from samples where cyc_count > 1500 and insn_count < 10; 6|1507 sqlite> select insn_count,cyc_count from samples where cyc_count > 1500; 118|2210 140|1516 3783|1861 132|1521 6|1507 sqlite> Seems to work :-) Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190520113728.14389-17-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-05-20 14:37:22 +03:00
perf scripts python: export-to-sqlite.py: Export IPC information Export cycle and instruction counts on samples and calls tables. Committer testing: First runs some workload collecting intel_pt with the 'cyc' ter just for userspace: [root@quaco adrian.hunter]# perf record -o simple-retpoline.perf.data -e intel_pt/cyc/u ./simple-retpoline [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.035 MB simple-retpoline.perf.data ] [root@quaco adrian.hunter]# Then use the export-to-sqlite.py script to see if the changes in this cset don't make it to break and if the changes in the db schema are the ones expected: [root@quaco adrian.hunter]# perf script -i simple-retpoline.perf.data --itrace=be -s ~acme/libexec/perf-core/scripts/python/export-to-sqlite.py simple-retpoline.db branches calls 2019-05-31 11:50:46.942710 Creating database ... 2019-05-31 11:50:46.949663 Writing records... 2019-05-31 11:50:47.224033 Adding indexes 2019-05-31 11:50:47.231599 Done [root@quaco adrian.hunter]# Now lets use the db: [root@quaco adrian.hunter]# sqlite3 simple-retpoline.db SQLite version 3.26.0 2018-12-01 12:34:55 Enter ".help" for usage hints. sqlite> .schema samples CREATE TABLE samples (id integer NOT NULL PRIMARY KEY,evsel_id bigint,machine_id bigint,thread_id bigint,comm_id bigint,dso_id bigint,symbol_id bigint,sym_offset bigint,ip bigint,time bigint,cpuinteger,to_dso_id bigint,to_symbol_id bigint,to_sym_offset bigint,to_ip bigint,branch_type integer,in_tx boolean,call_path_id bigint,insn_count bigint,cyc_count bigint); sqlite> Cool, the 'insn_count' and 'cyc_count' are there, now lets see if we can use them in a query: sqlite> select insn_count,cyc_count from samples where cyc_count > 1500 and insn_count < 10; 6|1507 sqlite> select insn_count,cyc_count from samples where cyc_count > 1500; 118|2210 140|1516 3783|1861 132|1521 6|1507 sqlite> Seems to work :-) Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190520113728.14389-17-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-05-20 14:37:22 +03:00
perf scripts python: export-to-sqlite.py: Export IPC information Export cycle and instruction counts on samples and calls tables. Committer testing: First runs some workload collecting intel_pt with the 'cyc' ter just for userspace: [root@quaco adrian.hunter]# perf record -o simple-retpoline.perf.data -e intel_pt/cyc/u ./simple-retpoline [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.035 MB simple-retpoline.perf.data ] [root@quaco adrian.hunter]# Then use the export-to-sqlite.py script to see if the changes in this cset don't make it to break and if the changes in the db schema are the ones expected: [root@quaco adrian.hunter]# perf script -i simple-retpoline.perf.data --itrace=be -s ~acme/libexec/perf-core/scripts/python/export-to-sqlite.py simple-retpoline.db branches calls 2019-05-31 11:50:46.942710 Creating database ... 2019-05-31 11:50:46.949663 Writing records... 2019-05-31 11:50:47.224033 Adding indexes 2019-05-31 11:50:47.231599 Done [root@quaco adrian.hunter]# Now lets use the db: [root@quaco adrian.hunter]# sqlite3 simple-retpoline.db SQLite version 3.26.0 2018-12-01 12:34:55 Enter ".help" for usage hints. sqlite> .schema samples CREATE TABLE samples (id integer NOT NULL PRIMARY KEY,evsel_id bigint,machine_id bigint,thread_id bigint,comm_id bigint,dso_id bigint,symbol_id bigint,sym_offset bigint,ip bigint,time bigint,cpuinteger,to_dso_id bigint,to_symbol_id bigint,to_sym_offset bigint,to_ip bigint,branch_type integer,in_tx boolean,call_path_id bigint,insn_count bigint,cyc_count bigint); sqlite> Cool, the 'insn_count' and 'cyc_count' are there, now lets see if we can use them in a query: sqlite> select insn_count,cyc_count from samples where cyc_count > 1500 and insn_count < 10; 6|1507 sqlite> select insn_count,cyc_count from samples where cyc_count > 1500; 118|2210 140|1516 3783|1861 132|1521 6|1507 sqlite> Seems to work :-) Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190520113728.14389-17-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-05-20 14:37:22 +03:00
perf scripts python: export-to-sqlite.py: Export IPC information Export cycle and instruction counts on samples and calls tables. Committer testing: First runs some workload collecting intel_pt with the 'cyc' ter just for userspace: [root@quaco adrian.hunter]# perf record -o simple-retpoline.perf.data -e intel_pt/cyc/u ./simple-retpoline [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.035 MB simple-retpoline.perf.data ] [root@quaco adrian.hunter]# Then use the export-to-sqlite.py script to see if the changes in this cset don't make it to break and if the changes in the db schema are the ones expected: [root@quaco adrian.hunter]# perf script -i simple-retpoline.perf.data --itrace=be -s ~acme/libexec/perf-core/scripts/python/export-to-sqlite.py simple-retpoline.db branches calls 2019-05-31 11:50:46.942710 Creating database ... 2019-05-31 11:50:46.949663 Writing records... 2019-05-31 11:50:47.224033 Adding indexes 2019-05-31 11:50:47.231599 Done [root@quaco adrian.hunter]# Now lets use the db: [root@quaco adrian.hunter]# sqlite3 simple-retpoline.db SQLite version 3.26.0 2018-12-01 12:34:55 Enter ".help" for usage hints. sqlite> .schema samples CREATE TABLE samples (id integer NOT NULL PRIMARY KEY,evsel_id bigint,machine_id bigint,thread_id bigint,comm_id bigint,dso_id bigint,symbol_id bigint,sym_offset bigint,ip bigint,time bigint,cpuinteger,to_dso_id bigint,to_symbol_id bigint,to_sym_offset bigint,to_ip bigint,branch_type integer,in_tx boolean,call_path_id bigint,insn_count bigint,cyc_count bigint); sqlite> Cool, the 'insn_count' and 'cyc_count' are there, now lets see if we can use them in a query: sqlite> select insn_count,cyc_count from samples where cyc_count > 1500 and insn_count < 10; 6|1507 sqlite> select insn_count,cyc_count from samples where cyc_count > 1500; 118|2210 140|1516 3783|1861 132|1521 6|1507 sqlite> Seems to work :-) Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190520113728.14389-17-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-05-20 14:37:22 +03:00