kernel/linux
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ synced 2026-10-16 22:39:48 +00:00
Code Issues Releases Packages Activity
linux/tools/perf/scripts/python/export-to-postgresql.py

1114 lines
39 KiB
Raw Normal View History

perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script python: Add Python3 support to export-to-postgresql.py Support both Python2 and Python3 in the export-to-postgresql.py script. The use of 'from __future__' implies the minimum supported Python2 version is now v2.6 Signed-off-by: Tony Jones <tonyj@suse.de> Link: http://lkml.kernel.org/r/20190309000518.2438-3-tonyj@suse.de Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Seeteena Thoufeek <s1seetee@linux.vnet.ibm.com> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-08 16:05:16 -08:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf tools: Add example call-graph script Add a script to produce a call-graph from data exported to a postgresql database and derived from a processor trace event like intel_pt or intel_bts. Refer to comments in the scripts call-graph-from-postgresql.py and export-to-postgresql.py for more details on how to set up the environment, install the required packages, etc. Committer note: From the scripts, for convenience while reading 'git log': An example of using this script with Intel PT: $ perf record -e intel_pt//u ls $ perf script -s ~/libexec/perf-core/scripts/python/export-to-postgresql.py pt_example branches calls 2015-05-29 12:49:23.464364 Creating database... 2015-05-29 12:49:26.281717 Writing to intermediate files... 2015-05-29 12:49:27.190383 Copying to database... 2015-05-29 12:49:28.140451 Removing intermediate files... 2015-05-29 12:49:28.147451 Adding primary keys 2015-05-29 12:49:28.655683 Adding foreign keys 2015-05-29 12:49:29.365350 Done $ python tools/perf/scripts/python/call-graph-from-postgresql.py pt_example # The result is a GUI window with a tree representing a context-sensitive # call-graph. Expanding a couple of levels of the tree and adjusting column # widths to suit will display something like: Call Graph: pt_example Call Path |Object |Count|Time(ns)|Time(%)|Branch Count|Branch Count(%) v- ls v- 2638:2638 v- _start ld-2.19.so 1 10074071 100.0 211135 100.0 |- unknown unknown 1 13198 0.1 1 0.0 >- _dl_start ld-2.19.so 1 1400980 13.9 19637 9.3 >- _d_linit_internal ld-2.19.so 1 448152 4.4 11094 5.3 v-__libc_start_main@plt ls 1 8211741 81.5 180397 85.4 >- _dl_fixup ld-2.19.so 1 7607 0.1 108 0.1 >- __cxa_atexit libc-2.19.so 1 11737 0.1 10 0.0 >- __libc_csu_init ls 1 10354 0.1 10 0.0 |- _setjmp libc-2.19.so 1 0 0.0 4 0.0 v- main ls 1 8182043 99.6 180254 99.9 Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/1437150840-31811-11-git-send-email-adrian.hunter@intel.com [ Added 'python-pyside qt-postgresql' to the yum cmdline installing required packages ] Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-07-17 19:33:45 +03:00
perf scripts python: export-to-postgresql.py: Add support for pyside2 pyside2 is the future for pyside support. Note pyside use Qt4 whereas pyside2 uses Qt5. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190412113830.4126-6-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-12 14:38:27 +03:00
perf tools: Add example call-graph script Add a script to produce a call-graph from data exported to a postgresql database and derived from a processor trace event like intel_pt or intel_bts. Refer to comments in the scripts call-graph-from-postgresql.py and export-to-postgresql.py for more details on how to set up the environment, install the required packages, etc. Committer note: From the scripts, for convenience while reading 'git log': An example of using this script with Intel PT: $ perf record -e intel_pt//u ls $ perf script -s ~/libexec/perf-core/scripts/python/export-to-postgresql.py pt_example branches calls 2015-05-29 12:49:23.464364 Creating database... 2015-05-29 12:49:26.281717 Writing to intermediate files... 2015-05-29 12:49:27.190383 Copying to database... 2015-05-29 12:49:28.140451 Removing intermediate files... 2015-05-29 12:49:28.147451 Adding primary keys 2015-05-29 12:49:28.655683 Adding foreign keys 2015-05-29 12:49:29.365350 Done $ python tools/perf/scripts/python/call-graph-from-postgresql.py pt_example # The result is a GUI window with a tree representing a context-sensitive # call-graph. Expanding a couple of levels of the tree and adjusting column # widths to suit will display something like: Call Graph: pt_example Call Path |Object |Count|Time(ns)|Time(%)|Branch Count|Branch Count(%) v- ls v- 2638:2638 v- _start ld-2.19.so 1 10074071 100.0 211135 100.0 |- unknown unknown 1 13198 0.1 1 0.0 >- _dl_start ld-2.19.so 1 1400980 13.9 19637 9.3 >- _d_linit_internal ld-2.19.so 1 448152 4.4 11094 5.3 v-__libc_start_main@plt ls 1 8211741 81.5 180397 85.4 >- _dl_fixup ld-2.19.so 1 7607 0.1 108 0.1 >- __cxa_atexit libc-2.19.so 1 11737 0.1 10 0.0 >- __libc_csu_init ls 1 10354 0.1 10 0.0 |- _setjmp libc-2.19.so 1 0 0.0 4 0.0 v- main ls 1 8182043 99.6 180254 99.9 Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/1437150840-31811-11-git-send-email-adrian.hunter@intel.com [ Added 'python-pyside qt-postgresql' to the yum cmdline installing required packages ] Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-07-17 19:33:45 +03:00
perf scripts python: export-to-postgresql.py: Add support for pyside2 pyside2 is the future for pyside support. Note pyside use Qt4 whereas pyside2 uses Qt5. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190412113830.4126-6-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-12 14:38:27 +03:00
perf tools: Add example call-graph script Add a script to produce a call-graph from data exported to a postgresql database and derived from a processor trace event like intel_pt or intel_bts. Refer to comments in the scripts call-graph-from-postgresql.py and export-to-postgresql.py for more details on how to set up the environment, install the required packages, etc. Committer note: From the scripts, for convenience while reading 'git log': An example of using this script with Intel PT: $ perf record -e intel_pt//u ls $ perf script -s ~/libexec/perf-core/scripts/python/export-to-postgresql.py pt_example branches calls 2015-05-29 12:49:23.464364 Creating database... 2015-05-29 12:49:26.281717 Writing to intermediate files... 2015-05-29 12:49:27.190383 Copying to database... 2015-05-29 12:49:28.140451 Removing intermediate files... 2015-05-29 12:49:28.147451 Adding primary keys 2015-05-29 12:49:28.655683 Adding foreign keys 2015-05-29 12:49:29.365350 Done $ python tools/perf/scripts/python/call-graph-from-postgresql.py pt_example # The result is a GUI window with a tree representing a context-sensitive # call-graph. Expanding a couple of levels of the tree and adjusting column # widths to suit will display something like: Call Graph: pt_example Call Path |Object |Count|Time(ns)|Time(%)|Branch Count|Branch Count(%) v- ls v- 2638:2638 v- _start ld-2.19.so 1 10074071 100.0 211135 100.0 |- unknown unknown 1 13198 0.1 1 0.0 >- _dl_start ld-2.19.so 1 1400980 13.9 19637 9.3 >- _d_linit_internal ld-2.19.so 1 448152 4.4 11094 5.3 v-__libc_start_main@plt ls 1 8211741 81.5 180397 85.4 >- _dl_fixup ld-2.19.so 1 7607 0.1 108 0.1 >- __cxa_atexit libc-2.19.so 1 11737 0.1 10 0.0 >- __libc_csu_init ls 1 10354 0.1 10 0.0 |- _setjmp libc-2.19.so 1 0 0.0 4 0.0 v- main ls 1 8182043 99.6 180254 99.9 Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/1437150840-31811-11-git-send-email-adrian.hunter@intel.com [ Added 'python-pyside qt-postgresql' to the yum cmdline installing required packages ] Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-07-17 19:33:45 +03:00
perf scripts python: export-to-postgresql.py: Add support for pyside2 pyside2 is the future for pyside support. Note pyside use Qt4 whereas pyside2 uses Qt5. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190412113830.4126-6-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-12 14:38:27 +03:00
perf tools: Add example call-graph script Add a script to produce a call-graph from data exported to a postgresql database and derived from a processor trace event like intel_pt or intel_bts. Refer to comments in the scripts call-graph-from-postgresql.py and export-to-postgresql.py for more details on how to set up the environment, install the required packages, etc. Committer note: From the scripts, for convenience while reading 'git log': An example of using this script with Intel PT: $ perf record -e intel_pt//u ls $ perf script -s ~/libexec/perf-core/scripts/python/export-to-postgresql.py pt_example branches calls 2015-05-29 12:49:23.464364 Creating database... 2015-05-29 12:49:26.281717 Writing to intermediate files... 2015-05-29 12:49:27.190383 Copying to database... 2015-05-29 12:49:28.140451 Removing intermediate files... 2015-05-29 12:49:28.147451 Adding primary keys 2015-05-29 12:49:28.655683 Adding foreign keys 2015-05-29 12:49:29.365350 Done $ python tools/perf/scripts/python/call-graph-from-postgresql.py pt_example # The result is a GUI window with a tree representing a context-sensitive # call-graph. Expanding a couple of levels of the tree and adjusting column # widths to suit will display something like: Call Graph: pt_example Call Path |Object |Count|Time(ns)|Time(%)|Branch Count|Branch Count(%) v- ls v- 2638:2638 v- _start ld-2.19.so 1 10074071 100.0 211135 100.0 |- unknown unknown 1 13198 0.1 1 0.0 >- _dl_start ld-2.19.so 1 1400980 13.9 19637 9.3 >- _d_linit_internal ld-2.19.so 1 448152 4.4 11094 5.3 v-__libc_start_main@plt ls 1 8211741 81.5 180397 85.4 >- _dl_fixup ld-2.19.so 1 7607 0.1 108 0.1 >- __cxa_atexit libc-2.19.so 1 11737 0.1 10 0.0 >- __libc_csu_init ls 1 10354 0.1 10 0.0 |- _setjmp libc-2.19.so 1 0 0.0 4 0.0 v- main ls 1 8182043 99.6 180254 99.9 Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/1437150840-31811-11-git-send-email-adrian.hunter@intel.com [ Added 'python-pyside qt-postgresql' to the yum cmdline installing required packages ] Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-07-17 19:33:45 +03:00
perf scripts python: export-to-postgresql.py: Add support for pyside2 pyside2 is the future for pyside support. Note pyside use Qt4 whereas pyside2 uses Qt5. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190412113830.4126-6-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-12 14:38:27 +03:00
perf tools: Add example call-graph script Add a script to produce a call-graph from data exported to a postgresql database and derived from a processor trace event like intel_pt or intel_bts. Refer to comments in the scripts call-graph-from-postgresql.py and export-to-postgresql.py for more details on how to set up the environment, install the required packages, etc. Committer note: From the scripts, for convenience while reading 'git log': An example of using this script with Intel PT: $ perf record -e intel_pt//u ls $ perf script -s ~/libexec/perf-core/scripts/python/export-to-postgresql.py pt_example branches calls 2015-05-29 12:49:23.464364 Creating database... 2015-05-29 12:49:26.281717 Writing to intermediate files... 2015-05-29 12:49:27.190383 Copying to database... 2015-05-29 12:49:28.140451 Removing intermediate files... 2015-05-29 12:49:28.147451 Adding primary keys 2015-05-29 12:49:28.655683 Adding foreign keys 2015-05-29 12:49:29.365350 Done $ python tools/perf/scripts/python/call-graph-from-postgresql.py pt_example # The result is a GUI window with a tree representing a context-sensitive # call-graph. Expanding a couple of levels of the tree and adjusting column # widths to suit will display something like: Call Graph: pt_example Call Path |Object |Count|Time(ns)|Time(%)|Branch Count|Branch Count(%) v- ls v- 2638:2638 v- _start ld-2.19.so 1 10074071 100.0 211135 100.0 |- unknown unknown 1 13198 0.1 1 0.0 >- _dl_start ld-2.19.so 1 1400980 13.9 19637 9.3 >- _d_linit_internal ld-2.19.so 1 448152 4.4 11094 5.3 v-__libc_start_main@plt ls 1 8211741 81.5 180397 85.4 >- _dl_fixup ld-2.19.so 1 7607 0.1 108 0.1 >- __cxa_atexit libc-2.19.so 1 11737 0.1 10 0.0 >- __libc_csu_init ls 1 10354 0.1 10 0.0 |- _setjmp libc-2.19.so 1 0 0.0 4 0.0 v- main ls 1 8182043 99.6 180254 99.9 Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/1437150840-31811-11-git-send-email-adrian.hunter@intel.com [ Added 'python-pyside qt-postgresql' to the yum cmdline installing required packages ] Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-07-17 19:33:45 +03:00
perf script: Fix postgresql ubuntu install instructions The current instructions for setting up an Ubuntu system for using the export-to-postgresql.py script are incorrect. The instructions in the script have been updated to work on newer versions of ubuntu. -Add missing dependencies to apt-get command: python-pyside.qtsql, libqt4-sql-psql -Add '-s' option to createuser command to force the user to be a superuser since the command doesn't prompt as indicated in the current instructions. Tested on: Ubuntu 14.04, Ubuntu 16.04(beta) Signed-off-by: Chris Phlipot <cphlipot0@gmail.com> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1461056164-14914-3-git-send-email-cphlipot0@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-04-19 01:56:02 -07:00
perf scripts python: export-to-postgresql.py: Add support for pyside2 pyside2 is the future for pyside support. Note pyside use Qt4 whereas pyside2 uses Qt5. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190412113830.4126-6-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-12 14:38:27 +03:00
perf tools: Add example call-graph script Add a script to produce a call-graph from data exported to a postgresql database and derived from a processor trace event like intel_pt or intel_bts. Refer to comments in the scripts call-graph-from-postgresql.py and export-to-postgresql.py for more details on how to set up the environment, install the required packages, etc. Committer note: From the scripts, for convenience while reading 'git log': An example of using this script with Intel PT: $ perf record -e intel_pt//u ls $ perf script -s ~/libexec/perf-core/scripts/python/export-to-postgresql.py pt_example branches calls 2015-05-29 12:49:23.464364 Creating database... 2015-05-29 12:49:26.281717 Writing to intermediate files... 2015-05-29 12:49:27.190383 Copying to database... 2015-05-29 12:49:28.140451 Removing intermediate files... 2015-05-29 12:49:28.147451 Adding primary keys 2015-05-29 12:49:28.655683 Adding foreign keys 2015-05-29 12:49:29.365350 Done $ python tools/perf/scripts/python/call-graph-from-postgresql.py pt_example # The result is a GUI window with a tree representing a context-sensitive # call-graph. Expanding a couple of levels of the tree and adjusting column # widths to suit will display something like: Call Graph: pt_example Call Path |Object |Count|Time(ns)|Time(%)|Branch Count|Branch Count(%) v- ls v- 2638:2638 v- _start ld-2.19.so 1 10074071 100.0 211135 100.0 |- unknown unknown 1 13198 0.1 1 0.0 >- _dl_start ld-2.19.so 1 1400980 13.9 19637 9.3 >- _d_linit_internal ld-2.19.so 1 448152 4.4 11094 5.3 v-__libc_start_main@plt ls 1 8211741 81.5 180397 85.4 >- _dl_fixup ld-2.19.so 1 7607 0.1 108 0.1 >- __cxa_atexit libc-2.19.so 1 11737 0.1 10 0.0 >- __libc_csu_init ls 1 10354 0.1 10 0.0 |- _setjmp libc-2.19.so 1 0 0.0 4 0.0 v- main ls 1 8182043 99.6 180254 99.9 Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/1437150840-31811-11-git-send-email-adrian.hunter@intel.com [ Added 'python-pyside qt-postgresql' to the yum cmdline installing required packages ] Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-07-17 19:33:45 +03:00
perf scripts python: call-graph-from-sql.py: Rename to exported-sql-viewer.py Additional reports will be added to the script so rename to reflect the more general purpose. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Andi Kleen <ak@linux.intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20181001062853.28285-13-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-10-01 09:28:46 +03:00
perf tools: Add more documentation to export-to-postgresql.py script Add some comments to the script and some 'views' to the created database that better illustrate the database structure and how it can be used. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/1443186956-18718-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-09-25 16:15:38 +03:00
perf tools: Add example call-graph script Add a script to produce a call-graph from data exported to a postgresql database and derived from a processor trace event like intel_pt or intel_bts. Refer to comments in the scripts call-graph-from-postgresql.py and export-to-postgresql.py for more details on how to set up the environment, install the required packages, etc. Committer note: From the scripts, for convenience while reading 'git log': An example of using this script with Intel PT: $ perf record -e intel_pt//u ls $ perf script -s ~/libexec/perf-core/scripts/python/export-to-postgresql.py pt_example branches calls 2015-05-29 12:49:23.464364 Creating database... 2015-05-29 12:49:26.281717 Writing to intermediate files... 2015-05-29 12:49:27.190383 Copying to database... 2015-05-29 12:49:28.140451 Removing intermediate files... 2015-05-29 12:49:28.147451 Adding primary keys 2015-05-29 12:49:28.655683 Adding foreign keys 2015-05-29 12:49:29.365350 Done $ python tools/perf/scripts/python/call-graph-from-postgresql.py pt_example # The result is a GUI window with a tree representing a context-sensitive # call-graph. Expanding a couple of levels of the tree and adjusting column # widths to suit will display something like: Call Graph: pt_example Call Path |Object |Count|Time(ns)|Time(%)|Branch Count|Branch Count(%) v- ls v- 2638:2638 v- _start ld-2.19.so 1 10074071 100.0 211135 100.0 |- unknown unknown 1 13198 0.1 1 0.0 >- _dl_start ld-2.19.so 1 1400980 13.9 19637 9.3 >- _d_linit_internal ld-2.19.so 1 448152 4.4 11094 5.3 v-__libc_start_main@plt ls 1 8211741 81.5 180397 85.4 >- _dl_fixup ld-2.19.so 1 7607 0.1 108 0.1 >- __cxa_atexit libc-2.19.so 1 11737 0.1 10 0.0 >- __libc_csu_init ls 1 10354 0.1 10 0.0 |- _setjmp libc-2.19.so 1 0 0.0 4 0.0 v- main ls 1 8182043 99.6 180254 99.9 Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/1437150840-31811-11-git-send-email-adrian.hunter@intel.com [ Added 'python-pyside qt-postgresql' to the yum cmdline installing required packages ] Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-07-17 19:33:45 +03:00
perf scripts python: export-to-postgresql.py: Add support for pyside2 pyside2 is the future for pyside support. Note pyside use Qt4 whereas pyside2 uses Qt5. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190412113830.4126-6-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-12 14:38:27 +03:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script python: Add Python3 support to export-to-postgresql.py Support both Python2 and Python3 in the export-to-postgresql.py script. The use of 'from __future__' implies the minimum supported Python2 version is now v2.6 Signed-off-by: Tony Jones <tonyj@suse.de> Link: http://lkml.kernel.org/r/20190309000518.2438-3-tonyj@suse.de Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Seeteena Thoufeek <s1seetee@linux.vnet.ibm.com> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-08 16:05:16 -08:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script python: Fix export-to-postgresql.py occasional failure Occasional export failures were found to be caused by truncating 64-bit pointers to 32-bits. Fix by explicitly setting types for all ctype arguments and results. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: stable@vger.kernel.org Link: http://lkml.kernel.org/r/20180911114504.28516-2-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-09-11 14:45:03 +03:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script python: Fix export-to-postgresql.py occasional failure Occasional export failures were found to be caused by truncating 64-bit pointers to 32-bits. Fix by explicitly setting types for all ctype arguments and results. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: stable@vger.kernel.org Link: http://lkml.kernel.org/r/20180911114504.28516-2-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-09-11 14:45:03 +03:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script python: Fix export-to-postgresql.py occasional failure Occasional export failures were found to be caused by truncating 64-bit pointers to 32-bits. Fix by explicitly setting types for all ctype arguments and results. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: stable@vger.kernel.org Link: http://lkml.kernel.org/r/20180911114504.28516-2-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-09-11 14:45:03 +03:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script python: Fix export-to-postgresql.py occasional failure Occasional export failures were found to be caused by truncating 64-bit pointers to 32-bits. Fix by explicitly setting types for all ctype arguments and results. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: stable@vger.kernel.org Link: http://lkml.kernel.org/r/20180911114504.28516-2-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-09-11 14:45:03 +03:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script python: Fix export-to-postgresql.py occasional failure Occasional export failures were found to be caused by truncating 64-bit pointers to 32-bits. Fix by explicitly setting types for all ctype arguments and results. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: stable@vger.kernel.org Link: http://lkml.kernel.org/r/20180911114504.28516-2-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-09-11 14:45:03 +03:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script python: Fix export-to-postgresql.py occasional failure Occasional export failures were found to be caused by truncating 64-bit pointers to 32-bits. Fix by explicitly setting types for all ctype arguments and results. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: stable@vger.kernel.org Link: http://lkml.kernel.org/r/20180911114504.28516-2-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-09-11 14:45:03 +03:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script python: Fix export-to-postgresql.py occasional failure Occasional export failures were found to be caused by truncating 64-bit pointers to 32-bits. Fix by explicitly setting types for all ctype arguments and results. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: stable@vger.kernel.org Link: http://lkml.kernel.org/r/20180911114504.28516-2-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2018-09-11 14:45:03 +03:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf tools: Add call information to Python export Add the ability to export detailed information about paired calls and returns to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-7-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:47 +02:00
perf script: Update export-to-postgresql to support callchain export Update the export-to-postgresql.py to support the newly introduced callchain export. callchains are added into the existing call_paths table and can now be associated with samples when the "callpaths" commandline option is used with the script. Ex.: $ perf script -s export-to-postgresql.py example_db all callchains Includes the following changes to enable callchain export via the python export APIs: - Add the "callchains" commandline option, which is used to enable callchain export by setting the perf_db_export_callchains global - Add perf_db_export_callchains checks for call_path table creation and population. - Add call_path_id to samples_table to conform with the new API example usage and output using a small test app: test_app.c: volatile int x = 0; void inc_x_loop() { int i; for(i=0; i<100000000; i++) x++; } void a() { inc_x_loop(); } void b() { inc_x_loop(); } int main() { a(); b(); return 0; } example usage: $ gcc -g -O0 test_app.c $ perf record --call-graph=dwarf ./a.out [ perf record: Woken up 77 times to write data ] [ perf record: Captured and wrote 19.373 MB perf.data (2404 samples) ] $ perf script -s scripts/python/export-to-postgresql.py example_db all callchains $ psql example_db example_db=# SELECT (SELECT name FROM symbols WHERE id = cps.symbol_id) as symbol, (SELECT name FROM symbols WHERE id = (SELECT symbol_id from call_paths where id = cps.parent_id)) as parent_symbol, sum(period) as event_count FROM samples join call_paths as cps on call_path_id = cps.id GROUP BY cps.id,evsel_id ORDER BY event_count DESC LIMIT 5; symbol | parent_symbol | event_count ------------------+--------------------------+------------- inc_x_loop | a | 734250982 inc_x_loop | b | 731028057 unknown | unknown | 1335858 task_tick_fair | scheduler_tick | 1238842 update_wall_time | tick_do_update_jiffies64 | 650373 (5 rows) The above data shows total "self time" in cycles for each call path that was sampled. It is intended to demonstrate how it accounts separately for the two ways to reach the "inc_x_loop" function(via "a" and "b"). Recursive common table expressions can be used as well to get cumulative time spent in a function as well, but that is beyond the scope of this basic example. Signed-off-by: Chris Phlipot <cphlipot0@gmail.com> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1461831551-12213-7-git-send-email-cphlipot0@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-04-28 01:19:11 -07:00
perf script python: Add Python3 support to export-to-postgresql.py Support both Python2 and Python3 in the export-to-postgresql.py script. The use of 'from __future__' implies the minimum supported Python2 version is now v2.6 Signed-off-by: Tony Jones <tonyj@suse.de> Link: http://lkml.kernel.org/r/20190309000518.2438-3-tonyj@suse.de Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Seeteena Thoufeek <s1seetee@linux.vnet.ibm.com> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-08 16:05:16 -08:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script python: Add printdate function to SQL exporters Introduce a printdate function to eliminate the repetitive use of datetime.datetime.today() in the SQL exporting scripts. Signed-off-by: Tony Jones <tonyj@suse.de> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Link: http://lkml.kernel.org/r/20190309000518.2438-5-tonyj@suse.de Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-08 16:05:18 -08:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf scripts python: export-to-postgresql.py: Add support for pyside2 pyside2 is the future for pyside support. Note pyside use Qt4 whereas pyside2 uses Qt5. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190412113830.4126-6-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-12 14:38:27 +03:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script: Update export-to-postgresql to support callchain export Update the export-to-postgresql.py to support the newly introduced callchain export. callchains are added into the existing call_paths table and can now be associated with samples when the "callpaths" commandline option is used with the script. Ex.: $ perf script -s export-to-postgresql.py example_db all callchains Includes the following changes to enable callchain export via the python export APIs: - Add the "callchains" commandline option, which is used to enable callchain export by setting the perf_db_export_callchains global - Add perf_db_export_callchains checks for call_path table creation and population. - Add call_path_id to samples_table to conform with the new API example usage and output using a small test app: test_app.c: volatile int x = 0; void inc_x_loop() { int i; for(i=0; i<100000000; i++) x++; } void a() { inc_x_loop(); } void b() { inc_x_loop(); } int main() { a(); b(); return 0; } example usage: $ gcc -g -O0 test_app.c $ perf record --call-graph=dwarf ./a.out [ perf record: Woken up 77 times to write data ] [ perf record: Captured and wrote 19.373 MB perf.data (2404 samples) ] $ perf script -s scripts/python/export-to-postgresql.py example_db all callchains $ psql example_db example_db=# SELECT (SELECT name FROM symbols WHERE id = cps.symbol_id) as symbol, (SELECT name FROM symbols WHERE id = (SELECT symbol_id from call_paths where id = cps.parent_id)) as parent_symbol, sum(period) as event_count FROM samples join call_paths as cps on call_path_id = cps.id GROUP BY cps.id,evsel_id ORDER BY event_count DESC LIMIT 5; symbol | parent_symbol | event_count ------------------+--------------------------+------------- inc_x_loop | a | 734250982 inc_x_loop | b | 731028057 unknown | unknown | 1335858 task_tick_fair | scheduler_tick | 1238842 update_wall_time | tick_do_update_jiffies64 | 650373 (5 rows) The above data shows total "self time" in cycles for each call path that was sampled. It is intended to demonstrate how it accounts separately for the two ways to reach the "inc_x_loop" function(via "a" and "b"). Recursive common table expressions can be used as well to get cumulative time spent in a function as well, but that is beyond the scope of this basic example. Signed-off-by: Chris Phlipot <cphlipot0@gmail.com> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1461831551-12213-7-git-send-email-cphlipot0@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-04-28 01:19:11 -07:00
perf tools: Add call information to Python export Add the ability to export detailed information about paired calls and returns to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-7-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:47 +02:00
perf script: Update export-to-postgresql to support callchain export Update the export-to-postgresql.py to support the newly introduced callchain export. callchains are added into the existing call_paths table and can now be associated with samples when the "callpaths" commandline option is used with the script. Ex.: $ perf script -s export-to-postgresql.py example_db all callchains Includes the following changes to enable callchain export via the python export APIs: - Add the "callchains" commandline option, which is used to enable callchain export by setting the perf_db_export_callchains global - Add perf_db_export_callchains checks for call_path table creation and population. - Add call_path_id to samples_table to conform with the new API example usage and output using a small test app: test_app.c: volatile int x = 0; void inc_x_loop() { int i; for(i=0; i<100000000; i++) x++; } void a() { inc_x_loop(); } void b() { inc_x_loop(); } int main() { a(); b(); return 0; } example usage: $ gcc -g -O0 test_app.c $ perf record --call-graph=dwarf ./a.out [ perf record: Woken up 77 times to write data ] [ perf record: Captured and wrote 19.373 MB perf.data (2404 samples) ] $ perf script -s scripts/python/export-to-postgresql.py example_db all callchains $ psql example_db example_db=# SELECT (SELECT name FROM symbols WHERE id = cps.symbol_id) as symbol, (SELECT name FROM symbols WHERE id = (SELECT symbol_id from call_paths where id = cps.parent_id)) as parent_symbol, sum(period) as event_count FROM samples join call_paths as cps on call_path_id = cps.id GROUP BY cps.id,evsel_id ORDER BY event_count DESC LIMIT 5; symbol | parent_symbol | event_count ------------------+--------------------------+------------- inc_x_loop | a | 734250982 inc_x_loop | b | 731028057 unknown | unknown | 1335858 task_tick_fair | scheduler_tick | 1238842 update_wall_time | tick_do_update_jiffies64 | 650373 (5 rows) The above data shows total "self time" in cycles for each call path that was sampled. It is intended to demonstrate how it accounts separately for the two ways to reach the "inc_x_loop" function(via "a" and "b"). Recursive common table expressions can be used as well to get cumulative time spent in a function as well, but that is beyond the scope of this basic example. Signed-off-by: Chris Phlipot <cphlipot0@gmail.com> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1461831551-12213-7-git-send-email-cphlipot0@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-04-28 01:19:11 -07:00
perf scripts python: export-to-postgresql.py: Add support for pyside2 pyside2 is the future for pyside support. Note pyside use Qt4 whereas pyside2 uses Qt5. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190412113830.4126-6-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-12 14:38:27 +03:00
perf tools: Add call information to Python export Add the ability to export detailed information about paired calls and returns to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-7-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:47 +02:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script python: Add printdate function to SQL exporters Introduce a printdate function to eliminate the repetitive use of datetime.datetime.today() in the SQL exporting scripts. Signed-off-by: Tony Jones <tonyj@suse.de> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Link: http://lkml.kernel.org/r/20190309000518.2438-5-tonyj@suse.de Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-08 16:05:18 -08:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf scripts python: export-to-postgresql.py: Export comm details Add table columns for thread id, comm start time and exec flag. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190710085810.1650-11-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-07-10 11:57:59 +03:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf tools: Add branch_type and in_tx to Python export Add branch_type and in_tx to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-4-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:44 +02:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf tools: Add branch_type and in_tx to Python export Add branch_type and in_tx to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-4-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:44 +02:00
perf scripts python: Fix missing call_path_id in export-to-postgresql script The export does not work if only branches are exported because of a missing column in the samples table. Fix by adding the missing call_path_id. Fixes: 3521f3bc9dae ("perf script: Update export-to-postgresql to support callchain export") Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Link: http://lkml.kernel.org/r/1501749090-20357-2-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-08-03 11:31:26 +03:00
perf scripts python: export-to-postgresql.py: Export IPC information Export cycle and instruction counts on samples and calls tables. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190520113728.14389-18-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-05-20 14:37:23 +03:00
perf scripts python: export-to-postgresql.py: Export all sample flags Add sample flags to the PostgreSQL database definition and export. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com> Cc: Andi Kleen <ak@linux.intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: https://lore.kernel.org/r/20220124084201.2699795-25-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2022-01-24 10:42:00 +02:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf tools: Add branch_type and in_tx to Python export Add branch_type and in_tx to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-4-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:44 +02:00
perf script: Update export-to-postgresql to support callchain export Update the export-to-postgresql.py to support the newly introduced callchain export. callchains are added into the existing call_paths table and can now be associated with samples when the "callpaths" commandline option is used with the script. Ex.: $ perf script -s export-to-postgresql.py example_db all callchains Includes the following changes to enable callchain export via the python export APIs: - Add the "callchains" commandline option, which is used to enable callchain export by setting the perf_db_export_callchains global - Add perf_db_export_callchains checks for call_path table creation and population. - Add call_path_id to samples_table to conform with the new API example usage and output using a small test app: test_app.c: volatile int x = 0; void inc_x_loop() { int i; for(i=0; i<100000000; i++) x++; } void a() { inc_x_loop(); } void b() { inc_x_loop(); } int main() { a(); b(); return 0; } example usage: $ gcc -g -O0 test_app.c $ perf record --call-graph=dwarf ./a.out [ perf record: Woken up 77 times to write data ] [ perf record: Captured and wrote 19.373 MB perf.data (2404 samples) ] $ perf script -s scripts/python/export-to-postgresql.py example_db all callchains $ psql example_db example_db=# SELECT (SELECT name FROM symbols WHERE id = cps.symbol_id) as symbol, (SELECT name FROM symbols WHERE id = (SELECT symbol_id from call_paths where id = cps.parent_id)) as parent_symbol, sum(period) as event_count FROM samples join call_paths as cps on call_path_id = cps.id GROUP BY cps.id,evsel_id ORDER BY event_count DESC LIMIT 5; symbol | parent_symbol | event_count ------------------+--------------------------+------------- inc_x_loop | a | 734250982 inc_x_loop | b | 731028057 unknown | unknown | 1335858 task_tick_fair | scheduler_tick | 1238842 update_wall_time | tick_do_update_jiffies64 | 650373 (5 rows) The above data shows total "self time" in cycles for each call path that was sampled. It is intended to demonstrate how it accounts separately for the two ways to reach the "inc_x_loop" function(via "a" and "b"). Recursive common table expressions can be used as well to get cumulative time spent in a function as well, but that is beyond the scope of this basic example. Signed-off-by: Chris Phlipot <cphlipot0@gmail.com> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1461831551-12213-7-git-send-email-cphlipot0@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-04-28 01:19:11 -07:00
perf scripts python: export-to-postgresql.py: Export IPC information Export cycle and instruction counts on samples and calls tables. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190520113728.14389-18-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-05-20 14:37:23 +03:00
perf scripts python: export-to-postgresql.py: Export all sample flags Add sample flags to the PostgreSQL database definition and export. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com> Cc: Andi Kleen <ak@linux.intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: https://lore.kernel.org/r/20220124084201.2699795-25-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2022-01-24 10:42:00 +02:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script: Update export-to-postgresql to support callchain export Update the export-to-postgresql.py to support the newly introduced callchain export. callchains are added into the existing call_paths table and can now be associated with samples when the "callpaths" commandline option is used with the script. Ex.: $ perf script -s export-to-postgresql.py example_db all callchains Includes the following changes to enable callchain export via the python export APIs: - Add the "callchains" commandline option, which is used to enable callchain export by setting the perf_db_export_callchains global - Add perf_db_export_callchains checks for call_path table creation and population. - Add call_path_id to samples_table to conform with the new API example usage and output using a small test app: test_app.c: volatile int x = 0; void inc_x_loop() { int i; for(i=0; i<100000000; i++) x++; } void a() { inc_x_loop(); } void b() { inc_x_loop(); } int main() { a(); b(); return 0; } example usage: $ gcc -g -O0 test_app.c $ perf record --call-graph=dwarf ./a.out [ perf record: Woken up 77 times to write data ] [ perf record: Captured and wrote 19.373 MB perf.data (2404 samples) ] $ perf script -s scripts/python/export-to-postgresql.py example_db all callchains $ psql example_db example_db=# SELECT (SELECT name FROM symbols WHERE id = cps.symbol_id) as symbol, (SELECT name FROM symbols WHERE id = (SELECT symbol_id from call_paths where id = cps.parent_id)) as parent_symbol, sum(period) as event_count FROM samples join call_paths as cps on call_path_id = cps.id GROUP BY cps.id,evsel_id ORDER BY event_count DESC LIMIT 5; symbol | parent_symbol | event_count ------------------+--------------------------+------------- inc_x_loop | a | 734250982 inc_x_loop | b | 731028057 unknown | unknown | 1335858 task_tick_fair | scheduler_tick | 1238842 update_wall_time | tick_do_update_jiffies64 | 650373 (5 rows) The above data shows total "self time" in cycles for each call path that was sampled. It is intended to demonstrate how it accounts separately for the two ways to reach the "inc_x_loop" function(via "a" and "b"). Recursive common table expressions can be used as well to get cumulative time spent in a function as well, but that is beyond the scope of this basic example. Signed-off-by: Chris Phlipot <cphlipot0@gmail.com> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1461831551-12213-7-git-send-email-cphlipot0@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-04-28 01:19:11 -07:00
perf tools: Add call information to Python export Add the ability to export detailed information about paired calls and returns to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-7-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:47 +02:00
perf script: Update export-to-postgresql to support callchain export Update the export-to-postgresql.py to support the newly introduced callchain export. callchains are added into the existing call_paths table and can now be associated with samples when the "callpaths" commandline option is used with the script. Ex.: $ perf script -s export-to-postgresql.py example_db all callchains Includes the following changes to enable callchain export via the python export APIs: - Add the "callchains" commandline option, which is used to enable callchain export by setting the perf_db_export_callchains global - Add perf_db_export_callchains checks for call_path table creation and population. - Add call_path_id to samples_table to conform with the new API example usage and output using a small test app: test_app.c: volatile int x = 0; void inc_x_loop() { int i; for(i=0; i<100000000; i++) x++; } void a() { inc_x_loop(); } void b() { inc_x_loop(); } int main() { a(); b(); return 0; } example usage: $ gcc -g -O0 test_app.c $ perf record --call-graph=dwarf ./a.out [ perf record: Woken up 77 times to write data ] [ perf record: Captured and wrote 19.373 MB perf.data (2404 samples) ] $ perf script -s scripts/python/export-to-postgresql.py example_db all callchains $ psql example_db example_db=# SELECT (SELECT name FROM symbols WHERE id = cps.symbol_id) as symbol, (SELECT name FROM symbols WHERE id = (SELECT symbol_id from call_paths where id = cps.parent_id)) as parent_symbol, sum(period) as event_count FROM samples join call_paths as cps on call_path_id = cps.id GROUP BY cps.id,evsel_id ORDER BY event_count DESC LIMIT 5; symbol | parent_symbol | event_count ------------------+--------------------------+------------- inc_x_loop | a | 734250982 inc_x_loop | b | 731028057 unknown | unknown | 1335858 task_tick_fair | scheduler_tick | 1238842 update_wall_time | tick_do_update_jiffies64 | 650373 (5 rows) The above data shows total "self time" in cycles for each call path that was sampled. It is intended to demonstrate how it accounts separately for the two ways to reach the "inc_x_loop" function(via "a" and "b"). Recursive common table expressions can be used as well to get cumulative time spent in a function as well, but that is beyond the scope of this basic example. Signed-off-by: Chris Phlipot <cphlipot0@gmail.com> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1461831551-12213-7-git-send-email-cphlipot0@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-04-28 01:19:11 -07:00
perf tools: Add call information to Python export Add the ability to export detailed information about paired calls and returns to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-7-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:47 +02:00
perf scripts python: export-to-postgresql.py: Export calls parent_id Export to the 'calls' table the newly created 'parent_id' and create an index for it. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: https://lkml.kernel.org/n/tip-eybd6fnk6j9r7g643lsideoo@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-28 15:00:27 +02:00
perf scripts python: export-to-postgresql.py: Export IPC information Export cycle and instruction counts on samples and calls tables. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190520113728.14389-18-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-05-20 14:37:23 +03:00
perf tools: Add call information to Python export Add the ability to export detailed information about paired calls and returns to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-7-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:47 +02:00
perf scripts python: export-to-postgresql.py: Export Intel PT power and ptwrite events The format of synthesized events is determined by the attribute config. For the formats for Intel PT power and ptwrite events, create tables and populate them when the synth_data handler is called. If the tables remain empty, drop them at the end. The tables and views, including a combined power_events_view, will display automatically from the tables menu of the exported exported-sql-viewer.py script. Note, currently only Atoms since Gemini Lake have support for ptwrite and mwait, pwre, exstop and pwrx, but all Intel PT implementations support cbr. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190622093248.581-8-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-06-22 12:32:48 +03:00
perf scripts python: export-to-postgresql.py: Export switch events Export switch events to a new table 'context_switches' and create a view 'context_switches_view'. The table and view will show automatically in the exported-sql-viewer.py script. If the table ends up empty, then it and the view are dropped. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190710085810.1650-22-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-07-10 11:58:10 +03:00
perf tools: Add more documentation to export-to-postgresql.py script Add some comments to the script and some 'views' to the created database that better illustrate the database structure and how it can be used. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/1443186956-18718-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-09-25 16:15:38 +03:00
perf script: Update export-to-postgresql to support callchain export Update the export-to-postgresql.py to support the newly introduced callchain export. callchains are added into the existing call_paths table and can now be associated with samples when the "callpaths" commandline option is used with the script. Ex.: $ perf script -s export-to-postgresql.py example_db all callchains Includes the following changes to enable callchain export via the python export APIs: - Add the "callchains" commandline option, which is used to enable callchain export by setting the perf_db_export_callchains global - Add perf_db_export_callchains checks for call_path table creation and population. - Add call_path_id to samples_table to conform with the new API example usage and output using a small test app: test_app.c: volatile int x = 0; void inc_x_loop() { int i; for(i=0; i<100000000; i++) x++; } void a() { inc_x_loop(); } void b() { inc_x_loop(); } int main() { a(); b(); return 0; } example usage: $ gcc -g -O0 test_app.c $ perf record --call-graph=dwarf ./a.out [ perf record: Woken up 77 times to write data ] [ perf record: Captured and wrote 19.373 MB perf.data (2404 samples) ] $ perf script -s scripts/python/export-to-postgresql.py example_db all callchains $ psql example_db example_db=# SELECT (SELECT name FROM symbols WHERE id = cps.symbol_id) as symbol, (SELECT name FROM symbols WHERE id = (SELECT symbol_id from call_paths where id = cps.parent_id)) as parent_symbol, sum(period) as event_count FROM samples join call_paths as cps on call_path_id = cps.id GROUP BY cps.id,evsel_id ORDER BY event_count DESC LIMIT 5; symbol | parent_symbol | event_count ------------------+--------------------------+------------- inc_x_loop | a | 734250982 inc_x_loop | b | 731028057 unknown | unknown | 1335858 task_tick_fair | scheduler_tick | 1238842 update_wall_time | tick_do_update_jiffies64 | 650373 (5 rows) The above data shows total "self time" in cycles for each call path that was sampled. It is intended to demonstrate how it accounts separately for the two ways to reach the "inc_x_loop" function(via "a" and "b"). Recursive common table expressions can be used as well to get cumulative time spent in a function as well, but that is beyond the scope of this basic example. Signed-off-by: Chris Phlipot <cphlipot0@gmail.com> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1461831551-12213-7-git-send-email-cphlipot0@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-04-28 01:19:11 -07:00
perf tools: Add more documentation to export-to-postgresql.py script Add some comments to the script and some 'views' to the created database that better illustrate the database structure and how it can be used. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/1443186956-18718-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-09-25 16:15:38 +03:00
perf script: Update export-to-postgresql to support callchain export Update the export-to-postgresql.py to support the newly introduced callchain export. callchains are added into the existing call_paths table and can now be associated with samples when the "callpaths" commandline option is used with the script. Ex.: $ perf script -s export-to-postgresql.py example_db all callchains Includes the following changes to enable callchain export via the python export APIs: - Add the "callchains" commandline option, which is used to enable callchain export by setting the perf_db_export_callchains global - Add perf_db_export_callchains checks for call_path table creation and population. - Add call_path_id to samples_table to conform with the new API example usage and output using a small test app: test_app.c: volatile int x = 0; void inc_x_loop() { int i; for(i=0; i<100000000; i++) x++; } void a() { inc_x_loop(); } void b() { inc_x_loop(); } int main() { a(); b(); return 0; } example usage: $ gcc -g -O0 test_app.c $ perf record --call-graph=dwarf ./a.out [ perf record: Woken up 77 times to write data ] [ perf record: Captured and wrote 19.373 MB perf.data (2404 samples) ] $ perf script -s scripts/python/export-to-postgresql.py example_db all callchains $ psql example_db example_db=# SELECT (SELECT name FROM symbols WHERE id = cps.symbol_id) as symbol, (SELECT name FROM symbols WHERE id = (SELECT symbol_id from call_paths where id = cps.parent_id)) as parent_symbol, sum(period) as event_count FROM samples join call_paths as cps on call_path_id = cps.id GROUP BY cps.id,evsel_id ORDER BY event_count DESC LIMIT 5; symbol | parent_symbol | event_count ------------------+--------------------------+------------- inc_x_loop | a | 734250982 inc_x_loop | b | 731028057 unknown | unknown | 1335858 task_tick_fair | scheduler_tick | 1238842 update_wall_time | tick_do_update_jiffies64 | 650373 (5 rows) The above data shows total "self time" in cycles for each call path that was sampled. It is intended to demonstrate how it accounts separately for the two ways to reach the "inc_x_loop" function(via "a" and "b"). Recursive common table expressions can be used as well to get cumulative time spent in a function as well, but that is beyond the scope of this basic example. Signed-off-by: Chris Phlipot <cphlipot0@gmail.com> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1461831551-12213-7-git-send-email-cphlipot0@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-04-28 01:19:11 -07:00
perf tools: Add more documentation to export-to-postgresql.py script Add some comments to the script and some 'views' to the created database that better illustrate the database structure and how it can be used. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/1443186956-18718-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-09-25 16:15:38 +03:00
perf scripts python: export-to-postgresql.py: Export IPC information Export cycle and instruction counts on samples and calls tables. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190520113728.14389-18-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-05-20 14:37:23 +03:00
perf tools: Add more documentation to export-to-postgresql.py script Add some comments to the script and some 'views' to the created database that better illustrate the database structure and how it can be used. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/1443186956-18718-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-09-25 16:15:38 +03:00
perf scripts python: export-to-postgresql.py: Fix invalid input syntax for integer error Fix SQL query error "invalid input syntax for integer": Traceback (most recent call last): File "tools/perf/scripts/python/export-to-postgresql.py", line 465, in <module> do_query(query, 'CREATE VIEW calls_view AS ' File "tools/perf/scripts/python/export-to-postgresql.py", line 274, in do_query raise Exception("Query failed: " + q.lastError().text()) Exception: Query failed: ERROR: invalid input syntax for integer: "" LINE 1: ...ch_count,call_id,return_id,CASE WHEN flags=0 THEN '' WHEN fl... ^ (22P02) QPSQL: Unable to create query Error running python script tools/perf/scripts/python/export-to-postgresql.py Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Fixes: f08046cb3082 ("perf thread-stack: Represent jmps to the start of a different symbol") Link: https://lkml.kernel.org/n/tip-strfpdozrvg7bi1xzrivxzqt@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-28 15:00:26 +02:00
perf scripts python: export-to-postgresql.py: Export calls parent_id Export to the 'calls' table the newly created 'parent_id' and create an index for it. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: https://lkml.kernel.org/n/tip-eybd6fnk6j9r7g643lsideoo@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-28 15:00:27 +02:00
perf tools: Add more documentation to export-to-postgresql.py script Add some comments to the script and some 'views' to the created database that better illustrate the database structure and how it can be used. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/1443186956-18718-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2015-09-25 16:15:38 +03:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf tools: Add branch_type and in_tx to Python export Add branch_type and in_tx to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-4-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:44 +02:00
perf scripts python: export-to-postgresql.py: Export IPC information Export cycle and instruction counts on samples and calls tables. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190520113728.14389-18-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-05-20 14:37:23 +03:00
perf scripts python: export-to-postgresql.py: Export all sample flags Add sample flags to the PostgreSQL database definition and export. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com> Cc: Andi Kleen <ak@linux.intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: https://lore.kernel.org/r/20220124084201.2699795-25-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2022-01-24 10:42:00 +02:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf scripts python: export-to-postgresql.py: Export Intel PT power and ptwrite events The format of synthesized events is determined by the attribute config. For the formats for Intel PT power and ptwrite events, create tables and populate them when the synth_data handler is called. If the tables remain empty, drop them at the end. The tables and views, including a combined power_events_view, will display automatically from the tables menu of the exported exported-sql-viewer.py script. Note, currently only Atoms since Gemini Lake have support for ptwrite and mwait, pwre, exstop and pwrx, but all Intel PT implementations support cbr. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190622093248.581-8-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-06-22 12:32:48 +03:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf scripts python: export-to-postgresql.py: Export switch events Export switch events to a new table 'context_switches' and create a view 'context_switches_view'. The table and view will show automatically in the exported-sql-viewer.py script. If the table ends up empty, then it and the view are dropped. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190710085810.1650-22-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-07-10 11:58:10 +03:00
perf script python: Add Python3 support to export-to-postgresql.py Support both Python2 and Python3 in the export-to-postgresql.py script. The use of 'from __future__' implies the minimum supported Python2 version is now v2.6 Signed-off-by: Tony Jones <tonyj@suse.de> Link: http://lkml.kernel.org/r/20190309000518.2438-3-tonyj@suse.de Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Seeteena Thoufeek <s1seetee@linux.vnet.ibm.com> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-08 16:05:16 -08:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script python: Add Python3 support to export-to-postgresql.py Support both Python2 and Python3 in the export-to-postgresql.py script. The use of 'from __future__' implies the minimum supported Python2 version is now v2.6 Signed-off-by: Tony Jones <tonyj@suse.de> Link: http://lkml.kernel.org/r/20190309000518.2438-3-tonyj@suse.de Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Seeteena Thoufeek <s1seetee@linux.vnet.ibm.com> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-08 16:05:16 -08:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script python: Add Python3 support to export-to-postgresql.py Support both Python2 and Python3 in the export-to-postgresql.py script. The use of 'from __future__' implies the minimum supported Python2 version is now v2.6 Signed-off-by: Tony Jones <tonyj@suse.de> Link: http://lkml.kernel.org/r/20190309000518.2438-3-tonyj@suse.de Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Seeteena Thoufeek <s1seetee@linux.vnet.ibm.com> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-08 16:05:16 -08:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script python: Add Python3 support to export-to-postgresql.py Support both Python2 and Python3 in the export-to-postgresql.py script. The use of 'from __future__' implies the minimum supported Python2 version is now v2.6 Signed-off-by: Tony Jones <tonyj@suse.de> Link: http://lkml.kernel.org/r/20190309000518.2438-3-tonyj@suse.de Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Seeteena Thoufeek <s1seetee@linux.vnet.ibm.com> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-08 16:05:16 -08:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf tools: Add branch_type and in_tx to Python export Add branch_type and in_tx to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-4-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:44 +02:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script: Update export-to-postgresql to support callchain export Update the export-to-postgresql.py to support the newly introduced callchain export. callchains are added into the existing call_paths table and can now be associated with samples when the "callpaths" commandline option is used with the script. Ex.: $ perf script -s export-to-postgresql.py example_db all callchains Includes the following changes to enable callchain export via the python export APIs: - Add the "callchains" commandline option, which is used to enable callchain export by setting the perf_db_export_callchains global - Add perf_db_export_callchains checks for call_path table creation and population. - Add call_path_id to samples_table to conform with the new API example usage and output using a small test app: test_app.c: volatile int x = 0; void inc_x_loop() { int i; for(i=0; i<100000000; i++) x++; } void a() { inc_x_loop(); } void b() { inc_x_loop(); } int main() { a(); b(); return 0; } example usage: $ gcc -g -O0 test_app.c $ perf record --call-graph=dwarf ./a.out [ perf record: Woken up 77 times to write data ] [ perf record: Captured and wrote 19.373 MB perf.data (2404 samples) ] $ perf script -s scripts/python/export-to-postgresql.py example_db all callchains $ psql example_db example_db=# SELECT (SELECT name FROM symbols WHERE id = cps.symbol_id) as symbol, (SELECT name FROM symbols WHERE id = (SELECT symbol_id from call_paths where id = cps.parent_id)) as parent_symbol, sum(period) as event_count FROM samples join call_paths as cps on call_path_id = cps.id GROUP BY cps.id,evsel_id ORDER BY event_count DESC LIMIT 5; symbol | parent_symbol | event_count ------------------+--------------------------+------------- inc_x_loop | a | 734250982 inc_x_loop | b | 731028057 unknown | unknown | 1335858 task_tick_fair | scheduler_tick | 1238842 update_wall_time | tick_do_update_jiffies64 | 650373 (5 rows) The above data shows total "self time" in cycles for each call path that was sampled. It is intended to demonstrate how it accounts separately for the two ways to reach the "inc_x_loop" function(via "a" and "b"). Recursive common table expressions can be used as well to get cumulative time spent in a function as well, but that is beyond the scope of this basic example. Signed-off-by: Chris Phlipot <cphlipot0@gmail.com> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1461831551-12213-7-git-send-email-cphlipot0@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-04-28 01:19:11 -07:00
perf tools: Add call information to Python export Add the ability to export detailed information about paired calls and returns to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-7-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:47 +02:00
perf script: Update export-to-postgresql to support callchain export Update the export-to-postgresql.py to support the newly introduced callchain export. callchains are added into the existing call_paths table and can now be associated with samples when the "callpaths" commandline option is used with the script. Ex.: $ perf script -s export-to-postgresql.py example_db all callchains Includes the following changes to enable callchain export via the python export APIs: - Add the "callchains" commandline option, which is used to enable callchain export by setting the perf_db_export_callchains global - Add perf_db_export_callchains checks for call_path table creation and population. - Add call_path_id to samples_table to conform with the new API example usage and output using a small test app: test_app.c: volatile int x = 0; void inc_x_loop() { int i; for(i=0; i<100000000; i++) x++; } void a() { inc_x_loop(); } void b() { inc_x_loop(); } int main() { a(); b(); return 0; } example usage: $ gcc -g -O0 test_app.c $ perf record --call-graph=dwarf ./a.out [ perf record: Woken up 77 times to write data ] [ perf record: Captured and wrote 19.373 MB perf.data (2404 samples) ] $ perf script -s scripts/python/export-to-postgresql.py example_db all callchains $ psql example_db example_db=# SELECT (SELECT name FROM symbols WHERE id = cps.symbol_id) as symbol, (SELECT name FROM symbols WHERE id = (SELECT symbol_id from call_paths where id = cps.parent_id)) as parent_symbol, sum(period) as event_count FROM samples join call_paths as cps on call_path_id = cps.id GROUP BY cps.id,evsel_id ORDER BY event_count DESC LIMIT 5; symbol | parent_symbol | event_count ------------------+--------------------------+------------- inc_x_loop | a | 734250982 inc_x_loop | b | 731028057 unknown | unknown | 1335858 task_tick_fair | scheduler_tick | 1238842 update_wall_time | tick_do_update_jiffies64 | 650373 (5 rows) The above data shows total "self time" in cycles for each call path that was sampled. It is intended to demonstrate how it accounts separately for the two ways to reach the "inc_x_loop" function(via "a" and "b"). Recursive common table expressions can be used as well to get cumulative time spent in a function as well, but that is beyond the scope of this basic example. Signed-off-by: Chris Phlipot <cphlipot0@gmail.com> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1461831551-12213-7-git-send-email-cphlipot0@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-04-28 01:19:11 -07:00
perf tools: Add call information to Python export Add the ability to export detailed information about paired calls and returns to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-7-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:47 +02:00
perf scripts python: export-to-postgresql.py: Export Intel PT power and ptwrite events The format of synthesized events is determined by the attribute config. For the formats for Intel PT power and ptwrite events, create tables and populate them when the synth_data handler is called. If the tables remain empty, drop them at the end. The tables and views, including a combined power_events_view, will display automatically from the tables menu of the exported exported-sql-viewer.py script. Note, currently only Atoms since Gemini Lake have support for ptwrite and mwait, pwre, exstop and pwrx, but all Intel PT implementations support cbr. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190622093248.581-8-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-06-22 12:32:48 +03:00
perf scripts python: export-to-postgresql.py: Export switch events Export switch events to a new table 'context_switches' and create a view 'context_switches_view'. The table and view will show automatically in the exported-sql-viewer.py script. If the table ends up empty, then it and the view are dropped. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190710085810.1650-22-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-07-10 11:58:10 +03:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script python: Add printdate function to SQL exporters Introduce a printdate function to eliminate the repetitive use of datetime.datetime.today() in the SQL exporting scripts. Signed-off-by: Tony Jones <tonyj@suse.de> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Link: http://lkml.kernel.org/r/20190309000518.2438-5-tonyj@suse.de Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-08 16:05:18 -08:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf scripts python: export-to-postgresql.py: Export comm details Add table columns for thread id, comm start time and exec flag. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190710085810.1650-11-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-07-10 11:57:59 +03:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf scripts python: export-to-postgresql.py: Export all sample flags Add sample flags to the PostgreSQL database definition and export. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com> Cc: Andi Kleen <ak@linux.intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: https://lore.kernel.org/r/20220124084201.2699795-25-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2022-01-24 10:42:00 +02:00
perf script: Update export-to-postgresql to support callchain export Update the export-to-postgresql.py to support the newly introduced callchain export. callchains are added into the existing call_paths table and can now be associated with samples when the "callpaths" commandline option is used with the script. Ex.: $ perf script -s export-to-postgresql.py example_db all callchains Includes the following changes to enable callchain export via the python export APIs: - Add the "callchains" commandline option, which is used to enable callchain export by setting the perf_db_export_callchains global - Add perf_db_export_callchains checks for call_path table creation and population. - Add call_path_id to samples_table to conform with the new API example usage and output using a small test app: test_app.c: volatile int x = 0; void inc_x_loop() { int i; for(i=0; i<100000000; i++) x++; } void a() { inc_x_loop(); } void b() { inc_x_loop(); } int main() { a(); b(); return 0; } example usage: $ gcc -g -O0 test_app.c $ perf record --call-graph=dwarf ./a.out [ perf record: Woken up 77 times to write data ] [ perf record: Captured and wrote 19.373 MB perf.data (2404 samples) ] $ perf script -s scripts/python/export-to-postgresql.py example_db all callchains $ psql example_db example_db=# SELECT (SELECT name FROM symbols WHERE id = cps.symbol_id) as symbol, (SELECT name FROM symbols WHERE id = (SELECT symbol_id from call_paths where id = cps.parent_id)) as parent_symbol, sum(period) as event_count FROM samples join call_paths as cps on call_path_id = cps.id GROUP BY cps.id,evsel_id ORDER BY event_count DESC LIMIT 5; symbol | parent_symbol | event_count ------------------+--------------------------+------------- inc_x_loop | a | 734250982 inc_x_loop | b | 731028057 unknown | unknown | 1335858 task_tick_fair | scheduler_tick | 1238842 update_wall_time | tick_do_update_jiffies64 | 650373 (5 rows) The above data shows total "self time" in cycles for each call path that was sampled. It is intended to demonstrate how it accounts separately for the two ways to reach the "inc_x_loop" function(via "a" and "b"). Recursive common table expressions can be used as well to get cumulative time spent in a function as well, but that is beyond the scope of this basic example. Signed-off-by: Chris Phlipot <cphlipot0@gmail.com> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1461831551-12213-7-git-send-email-cphlipot0@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-04-28 01:19:11 -07:00
perf tools: Add call information to Python export Add the ability to export detailed information about paired calls and returns to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-7-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:47 +02:00
perf scripts python: export-to-postgresql.py: Export IPC information Export cycle and instruction counts on samples and calls tables. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190520113728.14389-18-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-05-20 14:37:23 +03:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf scripts python: export-to-postgresql.py: Export Intel PT power and ptwrite events The format of synthesized events is determined by the attribute config. For the formats for Intel PT power and ptwrite events, create tables and populate them when the synth_data handler is called. If the tables remain empty, drop them at the end. The tables and views, including a combined power_events_view, will display automatically from the tables menu of the exported exported-sql-viewer.py script. Note, currently only Atoms since Gemini Lake have support for ptwrite and mwait, pwre, exstop and pwrx, but all Intel PT implementations support cbr. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190622093248.581-8-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-06-22 12:32:48 +03:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script python: Add printdate function to SQL exporters Introduce a printdate function to eliminate the repetitive use of datetime.datetime.today() in the SQL exporting scripts. Signed-off-by: Tony Jones <tonyj@suse.de> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Link: http://lkml.kernel.org/r/20190309000518.2438-5-tonyj@suse.de Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-08 16:05:18 -08:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf tools: Add branch_type and in_tx to Python export Add branch_type and in_tx to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-4-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:44 +02:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script: Update export-to-postgresql to support callchain export Update the export-to-postgresql.py to support the newly introduced callchain export. callchains are added into the existing call_paths table and can now be associated with samples when the "callpaths" commandline option is used with the script. Ex.: $ perf script -s export-to-postgresql.py example_db all callchains Includes the following changes to enable callchain export via the python export APIs: - Add the "callchains" commandline option, which is used to enable callchain export by setting the perf_db_export_callchains global - Add perf_db_export_callchains checks for call_path table creation and population. - Add call_path_id to samples_table to conform with the new API example usage and output using a small test app: test_app.c: volatile int x = 0; void inc_x_loop() { int i; for(i=0; i<100000000; i++) x++; } void a() { inc_x_loop(); } void b() { inc_x_loop(); } int main() { a(); b(); return 0; } example usage: $ gcc -g -O0 test_app.c $ perf record --call-graph=dwarf ./a.out [ perf record: Woken up 77 times to write data ] [ perf record: Captured and wrote 19.373 MB perf.data (2404 samples) ] $ perf script -s scripts/python/export-to-postgresql.py example_db all callchains $ psql example_db example_db=# SELECT (SELECT name FROM symbols WHERE id = cps.symbol_id) as symbol, (SELECT name FROM symbols WHERE id = (SELECT symbol_id from call_paths where id = cps.parent_id)) as parent_symbol, sum(period) as event_count FROM samples join call_paths as cps on call_path_id = cps.id GROUP BY cps.id,evsel_id ORDER BY event_count DESC LIMIT 5; symbol | parent_symbol | event_count ------------------+--------------------------+------------- inc_x_loop | a | 734250982 inc_x_loop | b | 731028057 unknown | unknown | 1335858 task_tick_fair | scheduler_tick | 1238842 update_wall_time | tick_do_update_jiffies64 | 650373 (5 rows) The above data shows total "self time" in cycles for each call path that was sampled. It is intended to demonstrate how it accounts separately for the two ways to reach the "inc_x_loop" function(via "a" and "b"). Recursive common table expressions can be used as well to get cumulative time spent in a function as well, but that is beyond the scope of this basic example. Signed-off-by: Chris Phlipot <cphlipot0@gmail.com> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1461831551-12213-7-git-send-email-cphlipot0@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-04-28 01:19:11 -07:00
perf tools: Add call information to Python export Add the ability to export detailed information about paired calls and returns to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-7-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:47 +02:00
perf script: Update export-to-postgresql to support callchain export Update the export-to-postgresql.py to support the newly introduced callchain export. callchains are added into the existing call_paths table and can now be associated with samples when the "callpaths" commandline option is used with the script. Ex.: $ perf script -s export-to-postgresql.py example_db all callchains Includes the following changes to enable callchain export via the python export APIs: - Add the "callchains" commandline option, which is used to enable callchain export by setting the perf_db_export_callchains global - Add perf_db_export_callchains checks for call_path table creation and population. - Add call_path_id to samples_table to conform with the new API example usage and output using a small test app: test_app.c: volatile int x = 0; void inc_x_loop() { int i; for(i=0; i<100000000; i++) x++; } void a() { inc_x_loop(); } void b() { inc_x_loop(); } int main() { a(); b(); return 0; } example usage: $ gcc -g -O0 test_app.c $ perf record --call-graph=dwarf ./a.out [ perf record: Woken up 77 times to write data ] [ perf record: Captured and wrote 19.373 MB perf.data (2404 samples) ] $ perf script -s scripts/python/export-to-postgresql.py example_db all callchains $ psql example_db example_db=# SELECT (SELECT name FROM symbols WHERE id = cps.symbol_id) as symbol, (SELECT name FROM symbols WHERE id = (SELECT symbol_id from call_paths where id = cps.parent_id)) as parent_symbol, sum(period) as event_count FROM samples join call_paths as cps on call_path_id = cps.id GROUP BY cps.id,evsel_id ORDER BY event_count DESC LIMIT 5; symbol | parent_symbol | event_count ------------------+--------------------------+------------- inc_x_loop | a | 734250982 inc_x_loop | b | 731028057 unknown | unknown | 1335858 task_tick_fair | scheduler_tick | 1238842 update_wall_time | tick_do_update_jiffies64 | 650373 (5 rows) The above data shows total "self time" in cycles for each call path that was sampled. It is intended to demonstrate how it accounts separately for the two ways to reach the "inc_x_loop" function(via "a" and "b"). Recursive common table expressions can be used as well to get cumulative time spent in a function as well, but that is beyond the scope of this basic example. Signed-off-by: Chris Phlipot <cphlipot0@gmail.com> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1461831551-12213-7-git-send-email-cphlipot0@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-04-28 01:19:11 -07:00
perf tools: Add call information to Python export Add the ability to export detailed information about paired calls and returns to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-7-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:47 +02:00
perf scripts python: export-to-postgresql.py: Export Intel PT power and ptwrite events The format of synthesized events is determined by the attribute config. For the formats for Intel PT power and ptwrite events, create tables and populate them when the synth_data handler is called. If the tables remain empty, drop them at the end. The tables and views, including a combined power_events_view, will display automatically from the tables menu of the exported exported-sql-viewer.py script. Note, currently only Atoms since Gemini Lake have support for ptwrite and mwait, pwre, exstop and pwrx, but all Intel PT implementations support cbr. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190622093248.581-8-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-06-22 12:32:48 +03:00
perf scripts python: export-to-postgresql.py: Export switch events Export switch events to a new table 'context_switches' and create a view 'context_switches_view'. The table and view will show automatically in the exported-sql-viewer.py script. If the table ends up empty, then it and the view are dropped. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190710085810.1650-22-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-07-10 11:58:10 +03:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script python: Add printdate function to SQL exporters Introduce a printdate function to eliminate the repetitive use of datetime.datetime.today() in the SQL exporting scripts. Signed-off-by: Tony Jones <tonyj@suse.de> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Link: http://lkml.kernel.org/r/20190309000518.2438-5-tonyj@suse.de Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-08 16:05:18 -08:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf tools: Add branch_type and in_tx to Python export Add branch_type and in_tx to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-4-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:44 +02:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script: Update export-to-postgresql to support callchain export Update the export-to-postgresql.py to support the newly introduced callchain export. callchains are added into the existing call_paths table and can now be associated with samples when the "callpaths" commandline option is used with the script. Ex.: $ perf script -s export-to-postgresql.py example_db all callchains Includes the following changes to enable callchain export via the python export APIs: - Add the "callchains" commandline option, which is used to enable callchain export by setting the perf_db_export_callchains global - Add perf_db_export_callchains checks for call_path table creation and population. - Add call_path_id to samples_table to conform with the new API example usage and output using a small test app: test_app.c: volatile int x = 0; void inc_x_loop() { int i; for(i=0; i<100000000; i++) x++; } void a() { inc_x_loop(); } void b() { inc_x_loop(); } int main() { a(); b(); return 0; } example usage: $ gcc -g -O0 test_app.c $ perf record --call-graph=dwarf ./a.out [ perf record: Woken up 77 times to write data ] [ perf record: Captured and wrote 19.373 MB perf.data (2404 samples) ] $ perf script -s scripts/python/export-to-postgresql.py example_db all callchains $ psql example_db example_db=# SELECT (SELECT name FROM symbols WHERE id = cps.symbol_id) as symbol, (SELECT name FROM symbols WHERE id = (SELECT symbol_id from call_paths where id = cps.parent_id)) as parent_symbol, sum(period) as event_count FROM samples join call_paths as cps on call_path_id = cps.id GROUP BY cps.id,evsel_id ORDER BY event_count DESC LIMIT 5; symbol | parent_symbol | event_count ------------------+--------------------------+------------- inc_x_loop | a | 734250982 inc_x_loop | b | 731028057 unknown | unknown | 1335858 task_tick_fair | scheduler_tick | 1238842 update_wall_time | tick_do_update_jiffies64 | 650373 (5 rows) The above data shows total "self time" in cycles for each call path that was sampled. It is intended to demonstrate how it accounts separately for the two ways to reach the "inc_x_loop" function(via "a" and "b"). Recursive common table expressions can be used as well to get cumulative time spent in a function as well, but that is beyond the scope of this basic example. Signed-off-by: Chris Phlipot <cphlipot0@gmail.com> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1461831551-12213-7-git-send-email-cphlipot0@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-04-28 01:19:11 -07:00
perf tools: Add call information to Python export Add the ability to export detailed information about paired calls and returns to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-7-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:47 +02:00
perf script: Update export-to-postgresql to support callchain export Update the export-to-postgresql.py to support the newly introduced callchain export. callchains are added into the existing call_paths table and can now be associated with samples when the "callpaths" commandline option is used with the script. Ex.: $ perf script -s export-to-postgresql.py example_db all callchains Includes the following changes to enable callchain export via the python export APIs: - Add the "callchains" commandline option, which is used to enable callchain export by setting the perf_db_export_callchains global - Add perf_db_export_callchains checks for call_path table creation and population. - Add call_path_id to samples_table to conform with the new API example usage and output using a small test app: test_app.c: volatile int x = 0; void inc_x_loop() { int i; for(i=0; i<100000000; i++) x++; } void a() { inc_x_loop(); } void b() { inc_x_loop(); } int main() { a(); b(); return 0; } example usage: $ gcc -g -O0 test_app.c $ perf record --call-graph=dwarf ./a.out [ perf record: Woken up 77 times to write data ] [ perf record: Captured and wrote 19.373 MB perf.data (2404 samples) ] $ perf script -s scripts/python/export-to-postgresql.py example_db all callchains $ psql example_db example_db=# SELECT (SELECT name FROM symbols WHERE id = cps.symbol_id) as symbol, (SELECT name FROM symbols WHERE id = (SELECT symbol_id from call_paths where id = cps.parent_id)) as parent_symbol, sum(period) as event_count FROM samples join call_paths as cps on call_path_id = cps.id GROUP BY cps.id,evsel_id ORDER BY event_count DESC LIMIT 5; symbol | parent_symbol | event_count ------------------+--------------------------+------------- inc_x_loop | a | 734250982 inc_x_loop | b | 731028057 unknown | unknown | 1335858 task_tick_fair | scheduler_tick | 1238842 update_wall_time | tick_do_update_jiffies64 | 650373 (5 rows) The above data shows total "self time" in cycles for each call path that was sampled. It is intended to demonstrate how it accounts separately for the two ways to reach the "inc_x_loop" function(via "a" and "b"). Recursive common table expressions can be used as well to get cumulative time spent in a function as well, but that is beyond the scope of this basic example. Signed-off-by: Chris Phlipot <cphlipot0@gmail.com> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1461831551-12213-7-git-send-email-cphlipot0@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-04-28 01:19:11 -07:00
perf tools: Add call information to Python export Add the ability to export detailed information about paired calls and returns to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-7-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:47 +02:00
perf scripts python: export-to-postgresql.py: Export Intel PT power and ptwrite events The format of synthesized events is determined by the attribute config. For the formats for Intel PT power and ptwrite events, create tables and populate them when the synth_data handler is called. If the tables remain empty, drop them at the end. The tables and views, including a combined power_events_view, will display automatically from the tables menu of the exported exported-sql-viewer.py script. Note, currently only Atoms since Gemini Lake have support for ptwrite and mwait, pwre, exstop and pwrx, but all Intel PT implementations support cbr. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190622093248.581-8-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-06-22 12:32:48 +03:00
perf scripts python: export-to-postgresql.py: Export switch events Export switch events to a new table 'context_switches' and create a view 'context_switches_view'. The table and view will show automatically in the exported-sql-viewer.py script. If the table ends up empty, then it and the view are dropped. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190710085810.1650-22-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-07-10 11:58:10 +03:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script python: Add printdate function to SQL exporters Introduce a printdate function to eliminate the repetitive use of datetime.datetime.today() in the SQL exporting scripts. Signed-off-by: Tony Jones <tonyj@suse.de> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Link: http://lkml.kernel.org/r/20190309000518.2438-5-tonyj@suse.de Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-08 16:05:18 -08:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf tools: Add branch_type and in_tx to Python export Add branch_type and in_tx to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-4-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:44 +02:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script: Update export-to-postgresql to support callchain export Update the export-to-postgresql.py to support the newly introduced callchain export. callchains are added into the existing call_paths table and can now be associated with samples when the "callpaths" commandline option is used with the script. Ex.: $ perf script -s export-to-postgresql.py example_db all callchains Includes the following changes to enable callchain export via the python export APIs: - Add the "callchains" commandline option, which is used to enable callchain export by setting the perf_db_export_callchains global - Add perf_db_export_callchains checks for call_path table creation and population. - Add call_path_id to samples_table to conform with the new API example usage and output using a small test app: test_app.c: volatile int x = 0; void inc_x_loop() { int i; for(i=0; i<100000000; i++) x++; } void a() { inc_x_loop(); } void b() { inc_x_loop(); } int main() { a(); b(); return 0; } example usage: $ gcc -g -O0 test_app.c $ perf record --call-graph=dwarf ./a.out [ perf record: Woken up 77 times to write data ] [ perf record: Captured and wrote 19.373 MB perf.data (2404 samples) ] $ perf script -s scripts/python/export-to-postgresql.py example_db all callchains $ psql example_db example_db=# SELECT (SELECT name FROM symbols WHERE id = cps.symbol_id) as symbol, (SELECT name FROM symbols WHERE id = (SELECT symbol_id from call_paths where id = cps.parent_id)) as parent_symbol, sum(period) as event_count FROM samples join call_paths as cps on call_path_id = cps.id GROUP BY cps.id,evsel_id ORDER BY event_count DESC LIMIT 5; symbol | parent_symbol | event_count ------------------+--------------------------+------------- inc_x_loop | a | 734250982 inc_x_loop | b | 731028057 unknown | unknown | 1335858 task_tick_fair | scheduler_tick | 1238842 update_wall_time | tick_do_update_jiffies64 | 650373 (5 rows) The above data shows total "self time" in cycles for each call path that was sampled. It is intended to demonstrate how it accounts separately for the two ways to reach the "inc_x_loop" function(via "a" and "b"). Recursive common table expressions can be used as well to get cumulative time spent in a function as well, but that is beyond the scope of this basic example. Signed-off-by: Chris Phlipot <cphlipot0@gmail.com> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1461831551-12213-7-git-send-email-cphlipot0@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-04-28 01:19:11 -07:00
perf tools: Add call information to Python export Add the ability to export detailed information about paired calls and returns to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-7-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:47 +02:00
perf script: Update export-to-postgresql to support callchain export Update the export-to-postgresql.py to support the newly introduced callchain export. callchains are added into the existing call_paths table and can now be associated with samples when the "callpaths" commandline option is used with the script. Ex.: $ perf script -s export-to-postgresql.py example_db all callchains Includes the following changes to enable callchain export via the python export APIs: - Add the "callchains" commandline option, which is used to enable callchain export by setting the perf_db_export_callchains global - Add perf_db_export_callchains checks for call_path table creation and population. - Add call_path_id to samples_table to conform with the new API example usage and output using a small test app: test_app.c: volatile int x = 0; void inc_x_loop() { int i; for(i=0; i<100000000; i++) x++; } void a() { inc_x_loop(); } void b() { inc_x_loop(); } int main() { a(); b(); return 0; } example usage: $ gcc -g -O0 test_app.c $ perf record --call-graph=dwarf ./a.out [ perf record: Woken up 77 times to write data ] [ perf record: Captured and wrote 19.373 MB perf.data (2404 samples) ] $ perf script -s scripts/python/export-to-postgresql.py example_db all callchains $ psql example_db example_db=# SELECT (SELECT name FROM symbols WHERE id = cps.symbol_id) as symbol, (SELECT name FROM symbols WHERE id = (SELECT symbol_id from call_paths where id = cps.parent_id)) as parent_symbol, sum(period) as event_count FROM samples join call_paths as cps on call_path_id = cps.id GROUP BY cps.id,evsel_id ORDER BY event_count DESC LIMIT 5; symbol | parent_symbol | event_count ------------------+--------------------------+------------- inc_x_loop | a | 734250982 inc_x_loop | b | 731028057 unknown | unknown | 1335858 task_tick_fair | scheduler_tick | 1238842 update_wall_time | tick_do_update_jiffies64 | 650373 (5 rows) The above data shows total "self time" in cycles for each call path that was sampled. It is intended to demonstrate how it accounts separately for the two ways to reach the "inc_x_loop" function(via "a" and "b"). Recursive common table expressions can be used as well to get cumulative time spent in a function as well, but that is beyond the scope of this basic example. Signed-off-by: Chris Phlipot <cphlipot0@gmail.com> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1461831551-12213-7-git-send-email-cphlipot0@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-04-28 01:19:11 -07:00
perf tools: Add call information to Python export Add the ability to export detailed information about paired calls and returns to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-7-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:47 +02:00
perf scripts python: export-to-postgresql.py: Export Intel PT power and ptwrite events The format of synthesized events is determined by the attribute config. For the formats for Intel PT power and ptwrite events, create tables and populate them when the synth_data handler is called. If the tables remain empty, drop them at the end. The tables and views, including a combined power_events_view, will display automatically from the tables menu of the exported exported-sql-viewer.py script. Note, currently only Atoms since Gemini Lake have support for ptwrite and mwait, pwre, exstop and pwrx, but all Intel PT implementations support cbr. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190622093248.581-8-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-06-22 12:32:48 +03:00
perf scripts python: export-to-postgresql.py: Export switch events Export switch events to a new table 'context_switches' and create a view 'context_switches_view'. The table and view will show automatically in the exported-sql-viewer.py script. If the table ends up empty, then it and the view are dropped. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190710085810.1650-22-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-07-10 11:58:10 +03:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script python: Add printdate function to SQL exporters Introduce a printdate function to eliminate the repetitive use of datetime.datetime.today() in the SQL exporting scripts. Signed-off-by: Tony Jones <tonyj@suse.de> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Link: http://lkml.kernel.org/r/20190309000518.2438-5-tonyj@suse.de Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-08 16:05:18 -08:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf scripts python: export-to-postgresql.py: Export comm details Add table columns for thread id, comm start time and exec flag. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190710085810.1650-11-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-07-10 11:57:59 +03:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script: Update export-to-postgresql to support callchain export Update the export-to-postgresql.py to support the newly introduced callchain export. callchains are added into the existing call_paths table and can now be associated with samples when the "callpaths" commandline option is used with the script. Ex.: $ perf script -s export-to-postgresql.py example_db all callchains Includes the following changes to enable callchain export via the python export APIs: - Add the "callchains" commandline option, which is used to enable callchain export by setting the perf_db_export_callchains global - Add perf_db_export_callchains checks for call_path table creation and population. - Add call_path_id to samples_table to conform with the new API example usage and output using a small test app: test_app.c: volatile int x = 0; void inc_x_loop() { int i; for(i=0; i<100000000; i++) x++; } void a() { inc_x_loop(); } void b() { inc_x_loop(); } int main() { a(); b(); return 0; } example usage: $ gcc -g -O0 test_app.c $ perf record --call-graph=dwarf ./a.out [ perf record: Woken up 77 times to write data ] [ perf record: Captured and wrote 19.373 MB perf.data (2404 samples) ] $ perf script -s scripts/python/export-to-postgresql.py example_db all callchains $ psql example_db example_db=# SELECT (SELECT name FROM symbols WHERE id = cps.symbol_id) as symbol, (SELECT name FROM symbols WHERE id = (SELECT symbol_id from call_paths where id = cps.parent_id)) as parent_symbol, sum(period) as event_count FROM samples join call_paths as cps on call_path_id = cps.id GROUP BY cps.id,evsel_id ORDER BY event_count DESC LIMIT 5; symbol | parent_symbol | event_count ------------------+--------------------------+------------- inc_x_loop | a | 734250982 inc_x_loop | b | 731028057 unknown | unknown | 1335858 task_tick_fair | scheduler_tick | 1238842 update_wall_time | tick_do_update_jiffies64 | 650373 (5 rows) The above data shows total "self time" in cycles for each call path that was sampled. It is intended to demonstrate how it accounts separately for the two ways to reach the "inc_x_loop" function(via "a" and "b"). Recursive common table expressions can be used as well to get cumulative time spent in a function as well, but that is beyond the scope of this basic example. Signed-off-by: Chris Phlipot <cphlipot0@gmail.com> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1461831551-12213-7-git-send-email-cphlipot0@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-04-28 01:19:11 -07:00
perf tools: Add call information to Python export Add the ability to export detailed information about paired calls and returns to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-7-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:47 +02:00
perf script: Update export-to-postgresql to support callchain export Update the export-to-postgresql.py to support the newly introduced callchain export. callchains are added into the existing call_paths table and can now be associated with samples when the "callpaths" commandline option is used with the script. Ex.: $ perf script -s export-to-postgresql.py example_db all callchains Includes the following changes to enable callchain export via the python export APIs: - Add the "callchains" commandline option, which is used to enable callchain export by setting the perf_db_export_callchains global - Add perf_db_export_callchains checks for call_path table creation and population. - Add call_path_id to samples_table to conform with the new API example usage and output using a small test app: test_app.c: volatile int x = 0; void inc_x_loop() { int i; for(i=0; i<100000000; i++) x++; } void a() { inc_x_loop(); } void b() { inc_x_loop(); } int main() { a(); b(); return 0; } example usage: $ gcc -g -O0 test_app.c $ perf record --call-graph=dwarf ./a.out [ perf record: Woken up 77 times to write data ] [ perf record: Captured and wrote 19.373 MB perf.data (2404 samples) ] $ perf script -s scripts/python/export-to-postgresql.py example_db all callchains $ psql example_db example_db=# SELECT (SELECT name FROM symbols WHERE id = cps.symbol_id) as symbol, (SELECT name FROM symbols WHERE id = (SELECT symbol_id from call_paths where id = cps.parent_id)) as parent_symbol, sum(period) as event_count FROM samples join call_paths as cps on call_path_id = cps.id GROUP BY cps.id,evsel_id ORDER BY event_count DESC LIMIT 5; symbol | parent_symbol | event_count ------------------+--------------------------+------------- inc_x_loop | a | 734250982 inc_x_loop | b | 731028057 unknown | unknown | 1335858 task_tick_fair | scheduler_tick | 1238842 update_wall_time | tick_do_update_jiffies64 | 650373 (5 rows) The above data shows total "self time" in cycles for each call path that was sampled. It is intended to demonstrate how it accounts separately for the two ways to reach the "inc_x_loop" function(via "a" and "b"). Recursive common table expressions can be used as well to get cumulative time spent in a function as well, but that is beyond the scope of this basic example. Signed-off-by: Chris Phlipot <cphlipot0@gmail.com> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1461831551-12213-7-git-send-email-cphlipot0@gmail.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2016-04-28 01:19:11 -07:00
perf tools: Add call information to Python export Add the ability to export detailed information about paired calls and returns to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-7-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:47 +02:00
perf scripts python: export-to-postgresql.py: Export calls parent_id Export to the 'calls' table the newly created 'parent_id' and create an index for it. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: https://lkml.kernel.org/n/tip-eybd6fnk6j9r7g643lsideoo@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-02-28 15:00:27 +02:00
perf scripts python: export-to-postgresql.py: Add has_calls column to comms table Now that a thread's current comm is exported, it shows up in the call graph and call tree even if it has no calls. That can happen because the calls are recorded against the main thread's initial comm. Add a table column to make it easy for the exported-sql-viewer.py script to select only comms with calls. Committer testing: $ rm -f simple-retpoline.db $ sudo ~acme/bin/perf script -i simple-retpoline.perf.data --itrace=be -s ~/libexec/perf-core/scripts/python/export-to-sqlite.py simple-retpoline.db branches calls 2019-07-10 12:25:33.200529 Creating database ... 2019-07-10 12:25:33.211548 Writing records... 2019-07-10 12:25:33.549630 Adding indexes 2019-07-10 12:25:33.560715 Dropping unused tables 2019-07-10 12:25:33.580201 Done $ sha256sum tools/perf/scripts/python/export-to-sqlite.py ~/libexec/perf-core/scripts/python/export-to-sqlite.py 2922b642c392004dffa1d8789296478c85904623f5895bcb9b6cbf33e3ca999f tools/perf/scripts/python/export-to-sqlite.py 2922b642c392004dffa1d8789296478c85904623f5895bcb9b6cbf33e3ca999f /home/acme/libexec/perf-core/scripts/python/export-to-sqlite.py $ $ sqlite3 simple-retpoline.db SQLite version 3.26.0 2018-12-01 12:34:55 Enter ".help" for usage hints. sqlite> .schema comms CREATE TABLE comms (id integer NOT NULL PRIMARY KEY,comm varchar(16),c_thread_id bigint,c_time bigint,exec_flag boolean, has_calls boolean); sqlite> select id,has_calls from comms; 0|1 1|1 sqlite> select distinct comm_id from calls; 0 1 sqlite> Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190710085810.1650-15-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-07-10 11:58:03 +03:00
perf scripts python: export-to-postgresql.py: Export Intel PT power and ptwrite events The format of synthesized events is determined by the attribute config. For the formats for Intel PT power and ptwrite events, create tables and populate them when the synth_data handler is called. If the tables remain empty, drop them at the end. The tables and views, including a combined power_events_view, will display automatically from the tables menu of the exported exported-sql-viewer.py script. Note, currently only Atoms since Gemini Lake have support for ptwrite and mwait, pwre, exstop and pwrx, but all Intel PT implementations support cbr. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190622093248.581-8-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-06-22 12:32:48 +03:00
perf scripts python: export-to-postgresql.py: Export switch events Export switch events to a new table 'context_switches' and create a view 'context_switches_view'. The table and view will show automatically in the exported-sql-viewer.py script. If the table ends up empty, then it and the view are dropped. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190710085810.1650-22-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-07-10 11:58:10 +03:00
perf scripts python: export-to-postgresql.py: Export Intel PT power and ptwrite events The format of synthesized events is determined by the attribute config. For the formats for Intel PT power and ptwrite events, create tables and populate them when the synth_data handler is called. If the tables remain empty, drop them at the end. The tables and views, including a combined power_events_view, will display automatically from the tables menu of the exported exported-sql-viewer.py script. Note, currently only Atoms since Gemini Lake have support for ptwrite and mwait, pwre, exstop and pwrx, but all Intel PT implementations support cbr. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190622093248.581-8-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-06-22 12:32:48 +03:00
perf scripts python: export-to-postgresql.py: Fix DROP VIEW power_events_view PostgreSQL can error if power_events_view is not dropped before its dependent tables e.g. Exception: Query failed: ERROR: cannot drop table mwait because other objects depend on it DETAIL: view power_events_view depends on table mwait Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Fixes: aba44287a224 ("perf scripts python: export-to-postgresql.py: Export Intel PT power and ptwrite events") Link: http://lkml.kernel.org/r/20190708055232.5032-2-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-07-08 08:52:31 +03:00
perf scripts python: export-to-postgresql.py: Export Intel PT power and ptwrite events The format of synthesized events is determined by the attribute config. For the formats for Intel PT power and ptwrite events, create tables and populate them when the synth_data handler is called. If the tables remain empty, drop them at the end. The tables and views, including a combined power_events_view, will display automatically from the tables menu of the exported exported-sql-viewer.py script. Note, currently only Atoms since Gemini Lake have support for ptwrite and mwait, pwre, exstop and pwrx, but all Intel PT implementations support cbr. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190622093248.581-8-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-06-22 12:32:48 +03:00
perf scripts python: export-to-postgresql.py: Export switch events Export switch events to a new table 'context_switches' and create a view 'context_switches_view'. The table and view will show automatically in the exported-sql-viewer.py script. If the table ends up empty, then it and the view are dropped. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190710085810.1650-22-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-07-10 11:58:10 +03:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script python: Add printdate function to SQL exporters Introduce a printdate function to eliminate the repetitive use of datetime.datetime.today() in the SQL exporting scripts. Signed-off-by: Tony Jones <tonyj@suse.de> Acked-by: Adrian Hunter <adrian.hunter@intel.com> Link: http://lkml.kernel.org/r/20190309000518.2438-5-tonyj@suse.de Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-08 16:05:18 -08:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script python: Add Python3 support to export-to-postgresql.py Support both Python2 and Python3 in the export-to-postgresql.py script. The use of 'from __future__' implies the minimum supported Python2 version is now v2.6 Signed-off-by: Tony Jones <tonyj@suse.de> Link: http://lkml.kernel.org/r/20190309000518.2438-3-tonyj@suse.de Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Seeteena Thoufeek <s1seetee@linux.vnet.ibm.com> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-08 16:05:16 -08:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script python: Add Python3 support to export-to-postgresql.py Support both Python2 and Python3 in the export-to-postgresql.py script. The use of 'from __future__' implies the minimum supported Python2 version is now v2.6 Signed-off-by: Tony Jones <tonyj@suse.de> Link: http://lkml.kernel.org/r/20190309000518.2438-3-tonyj@suse.de Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Seeteena Thoufeek <s1seetee@linux.vnet.ibm.com> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-08 16:05:16 -08:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf scripts python: export-to-postgresql.py: Export comm details Add table columns for thread id, comm start time and exec flag. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190710085810.1650-11-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-07-10 11:57:59 +03:00
perf script python: Add Python3 support to export-to-postgresql.py Support both Python2 and Python3 in the export-to-postgresql.py script. The use of 'from __future__' implies the minimum supported Python2 version is now v2.6 Signed-off-by: Tony Jones <tonyj@suse.de> Link: http://lkml.kernel.org/r/20190309000518.2438-3-tonyj@suse.de Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Seeteena Thoufeek <s1seetee@linux.vnet.ibm.com> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-08 16:05:16 -08:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf scripts python: export-to-postgresql.py: Export comm details Add table columns for thread id, comm start time and exec flag. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190710085810.1650-11-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-07-10 11:57:59 +03:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script python: Add Python3 support to export-to-postgresql.py Support both Python2 and Python3 in the export-to-postgresql.py script. The use of 'from __future__' implies the minimum supported Python2 version is now v2.6 Signed-off-by: Tony Jones <tonyj@suse.de> Link: http://lkml.kernel.org/r/20190309000518.2438-3-tonyj@suse.de Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Seeteena Thoufeek <s1seetee@linux.vnet.ibm.com> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-08 16:05:16 -08:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf script python: Add Python3 support to export-to-postgresql.py Support both Python2 and Python3 in the export-to-postgresql.py script. The use of 'from __future__' implies the minimum supported Python2 version is now v2.6 Signed-off-by: Tony Jones <tonyj@suse.de> Link: http://lkml.kernel.org/r/20190309000518.2438-3-tonyj@suse.de Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Seeteena Thoufeek <s1seetee@linux.vnet.ibm.com> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-08 16:05:16 -08:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf tools: Add branch_type and in_tx to Python export Add branch_type and in_tx to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-4-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:44 +02:00
perf script python: Add Python3 support to export-to-postgresql.py Support both Python2 and Python3 in the export-to-postgresql.py script. The use of 'from __future__' implies the minimum supported Python2 version is now v2.6 Signed-off-by: Tony Jones <tonyj@suse.de> Link: http://lkml.kernel.org/r/20190309000518.2438-3-tonyj@suse.de Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Signed-off-by: Seeteena Thoufeek <s1seetee@linux.vnet.ibm.com> Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-03-08 16:05:16 -08:00
perf tools: Add branch_type and in_tx to Python export Add branch_type and in_tx to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-4-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:44 +02:00
perf scripts python: export-to-postgresql.py: Export all sample flags Add sample flags to the PostgreSQL database definition and export. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com> Cc: Andi Kleen <ak@linux.intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: https://lore.kernel.org/r/20220124084201.2699795-25-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2022-01-24 10:42:00 +02:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf scripts python: export-to-postgresql.py: Export all sample flags Add sample flags to the PostgreSQL database definition and export. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com> Cc: Andi Kleen <ak@linux.intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: https://lore.kernel.org/r/20220124084201.2699795-25-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2022-01-24 10:42:00 +02:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf scripts python: export-to-postgresql.py: Export all sample flags Add sample flags to the PostgreSQL database definition and export. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com> Cc: Andi Kleen <ak@linux.intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: https://lore.kernel.org/r/20220124084201.2699795-25-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2022-01-24 10:42:00 +02:00
perf script: Add Python script to export to postgresql Add a Python script to export to a postgresql database. The script requires the Python pyside module and the Qt PostgreSQL driver. The packages needed are probably named "python-pyside" and "libqt4-sql-psql" The caller of the script must be able to create postgresql databases. The script takes the database name as a parameter. The database and database tables are created. Data is written to flat files which are then imported using SQL COPY FROM. Example: $ perf record ls ... $ perf script report export-to-postgresql lsdb 2014-02-14 10:55:38.631431 Creating database... 2014-02-14 10:55:39.291958 Writing to intermediate files... 2014-02-14 10:55:39.350280 Copying to database... 2014-02-14 10:55:39.358536 Removing intermediate files... 2014-02-14 10:55:39.358665 Adding primary keys 2014-02-14 10:55:39.658697 Adding foreign keys 2014-02-14 10:55:39.667412 Done $ psql lsdb lsdb-# \d List of relations Schema | Name | Type | Owner --------+-----------------+-------+------- public | comm_threads | table | acme public | comms | table | acme public | dsos | table | acme public | machines | table | acme public | samples | table | acme public | samples_view | view | acme public | selected_events | table | acme public | symbols | table | acme public | threads | table | acme (9 rows) lsdb-# \d samples Table "public.samples" Column | Type | Modifiers ---------------+---------+----------- id | bigint | not null evsel_id | bigint | machine_id | bigint | thread_id | bigint | comm_id | bigint | dso_id | bigint | symbol_id | bigint | sym_offset | bigint | ip | bigint | time | bigint | cpu | integer | to_dso_id | bigint | to_symbol_id | bigint | to_sym_offset | bigint | to_ip | bigint | period | bigint | weight | bigint | transaction | bigint | data_src | bigint | Indexes: "samples_pkey" PRIMARY KEY, btree (id) Foreign-key constraints: "commfk" FOREIGN KEY (comm_id) REFERENCES comms(id) "dsofk" FOREIGN KEY (dso_id) REFERENCES dsos(id) "evselfk" FOREIGN KEY (evsel_id) REFERENCES selected_events(id) "machinefk" FOREIGN KEY (machine_id) REFERENCES machines(id) "symbolfk" FOREIGN KEY (symbol_id) REFERENCES symbols(id) "threadfk" FOREIGN KEY (thread_id) REFERENCES threads(id) "todsofk" FOREIGN KEY (to_dso_id) REFERENCES dsos(id) "tosymbolfk" FOREIGN KEY (to_symbol_id) REFERENCES symbols(id) lsdb-# \d samples_view View "public.samples_view" Column | Type | Modifiers -------------------+-------------------------+----------- id | bigint | time | bigint | cpu | integer | pid | integer | tid | integer | command | character varying(16) | event | character varying(80) | ip_hex | text | symbol | character varying(2048) | sym_offset | bigint | dso_short_name | character varying(256) | to_ip_hex | text | to_symbol | character varying(2048) | to_sym_offset | bigint | to_dso_short_name | character varying(256) | lsdb=# select * from samples_view; id| time |cpu | pid | tid |command| event | ip_hex | symbol |sym_off| dso_name|to_ip_hex|to_symbol|to_sym_off|to_dso_name --+------------+----+------+------+-------+--------+---------------+---------------------+-------+---------+---------+---------+----------+---------- 1 |12202825015 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 2 |12203258804 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown 3 |12203988119 | -1 | 7339 | 7339 |:17339 | cycles | fffff8104d24a |native_write_msr_safe| 10 | [kernel]| 0 | unknown | 0| unknown My notes (which may be out-of-date) on setting up postgresql so you can create databases: fedora: $ sudo yum install postgresql postgresql-server python-pyside qt-postgresql $ sudo su - postgres -c initdb $ sudo service postgresql start $ sudo su - postgres $ createuser -s <your username> I used the the unix user name in createuser. If it fails, try createuser without -s and answer the following question to allow your user to create tables: Shall the new role be a superuser? (y/n) y ubuntu: $ sudo apt-get install postgresql $ sudo su - postgres $ createuser <your username> Shall the new role be a superuser? (y/n) y You may want to disable automatic startup. One way is to edit /etc/postgresql/9.3/main/start.conf. Another is to disable the init script e.g. sudo update-rc.d postgresql disable Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414061124-26830-8-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-23 13:45:15 +03:00
perf tools: Add call information to Python export Add the ability to export detailed information about paired calls and returns to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-7-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:47 +02:00
perf scripts python: export-to-postgresql.py: Export IPC information Export cycle and instruction counts on samples and calls tables. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190520113728.14389-18-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-05-20 14:37:23 +03:00
perf tools: Add call information to Python export Add the ability to export detailed information about paired calls and returns to Python db export and the export-to-postgresql.py script. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@gmail.com> Cc: Paul Mackerras <paulus@samba.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Stephane Eranian <eranian@google.com> Link: http://lkml.kernel.org/r/1414678188-14946-7-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2014-10-30 16:09:47 +02:00
perf scripts python: export-to-postgresql.py: Export Intel PT power and ptwrite events The format of synthesized events is determined by the attribute config. For the formats for Intel PT power and ptwrite events, create tables and populate them when the synth_data handler is called. If the tables remain empty, drop them at the end. The tables and views, including a combined power_events_view, will display automatically from the tables menu of the exported exported-sql-viewer.py script. Note, currently only Atoms since Gemini Lake have support for ptwrite and mwait, pwre, exstop and pwrx, but all Intel PT implementations support cbr. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190622093248.581-8-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-06-22 12:32:48 +03:00
perf scripts python: export-to-postgresql.py: Fix struct.pack() int argument Python 3.8 is requiring that arguments being packed as integers are also integers. Add int() accordingly. Before: $ perf record -e intel_pt//u uname $ perf script --itrace=bep -s ~/libexec/perf-core/scripts/python/export-to-postgresql.py perf_data_db branches calls 2020-06-25 16:09:10.547256 Creating database... 2020-06-25 16:09:10.733185 Writing to intermediate files... Traceback (most recent call last): File "/home/ahunter/libexec/perf-core/scripts/python/export-to-postgresql.py", line 1106, in synth_data cbr(id, raw_buf) File "/home/ahunter/libexec/perf-core/scripts/python/export-to-postgresql.py", line 1058, in cbr value = struct.pack("!hiqiiiiii", 4, 8, id, 4, cbr, 4, MHz, 4, percent) struct.error: required argument is not an integer Fatal Python error: problem in Python trace event handler Python runtime state: initialized Current thread 0x00007f35d3695780 (most recent call first): <no Python frame> Aborted (core dumped) After: $ dropdb perf_data_db $ rm -rf perf_data_db-perf-data $ perf script --itrace=bep -s ~/libexec/perf-core/scripts/python/export-to-postgresql.py perf_data_db branches calls 2020-06-25 16:09:40.990267 Creating database... 2020-06-25 16:09:41.207009 Writing to intermediate files... 2020-06-25 16:09:41.270915 Copying to database... 2020-06-25 16:09:41.382030 Removing intermediate files... 2020-06-25 16:09:41.384630 Adding primary keys 2020-06-25 16:09:41.541894 Adding foreign keys 2020-06-25 16:09:41.677044 Dropping unused tables 2020-06-25 16:09:41.703761 Done Fixes: aba44287a224 ("perf scripts python: export-to-postgresql.py: Export Intel PT power and ptwrite events") Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: stable@vger.kernel.org Link: http://lore.kernel.org/lkml/20200629091955.17090-2-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2020-06-29 12:19:50 +03:00
perf scripts python: export-to-postgresql.py: Export Intel PT power and ptwrite events The format of synthesized events is determined by the attribute config. For the formats for Intel PT power and ptwrite events, create tables and populate them when the synth_data handler is called. If the tables remain empty, drop them at the end. The tables and views, including a combined power_events_view, will display automatically from the tables menu of the exported exported-sql-viewer.py script. Note, currently only Atoms since Gemini Lake have support for ptwrite and mwait, pwre, exstop and pwrx, but all Intel PT implementations support cbr. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190622093248.581-8-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-06-22 12:32:48 +03:00
perf scripts python: export-to-postgresql.py: Export switch events Export switch events to a new table 'context_switches' and create a view 'context_switches_view'. The table and view will show automatically in the exported-sql-viewer.py script. If the table ends up empty, then it and the view are dropped. Signed-off-by: Adrian Hunter <adrian.hunter@intel.com> Cc: Jiri Olsa <jolsa@redhat.com> Link: http://lkml.kernel.org/r/20190710085810.1650-22-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-07-10 11:58:10 +03:00