- 15 May, 2019 40 commits
-
-
Alexey Budankov authored
zstd_init(, comp_level = 0) initializes decompression part of API only hat now consists of zstd_decompress_stream() function. The perf.data PERF_RECORD_COMPRESSED records are decompressed using zstd_decompress_stream() function into a linked list of mmaped memory regions of mmap_comp_len size (struct decomp). After decompression of one COMPRESSED record its content is iterated and fetched for usual processing. The mmaped memory regions with decompressed events are kept in the linked list till the tool process termination. When dumping raw records (e.g., perf report -D --header) file offsets of events from compressed records are printed as zero. Committer notes: Since now we have support for processing PERF_RECORD_COMPRESSED, we see none, in raw form, like we saw in the previous patch commiter notes, they were decompressed into the usual PERF_RECORD_{FORK,MMAP,COMM,etc} records, we only see the stats for those PERF_RECORD_COMPRESSED events, and since I used the file generated in the commiter notes for the previous patch, there they are, 2 compressed records: $ perf report --header-only | grep cmdline # cmdline : /home/acme/bin/perf record -z2 sleep 1 $ perf report -D | grep COMPRESS COMPRESSED events: 2 COMPRESSED events: 0 $ perf report --stdio # To display the perf.data header info, please use --header/--header-only options. # # # Total Lost Samples: 0 # # Samples: 15 of event 'cycles:u' # Event count (approx.): 962227 # # Overhead Command Shared Object Symbol # ........ ....... ................ ........................... # 46.99% sleep libc-2.28.so [.] _dl_addr 29.24% sleep [unknown] [k] 0xffffffffaea00a67 16.45% sleep libc-2.28.so [.] __GI__IO_un_link.part.1 5.92% sleep ld-2.28.so [.] _dl_setup_hash 1.40% sleep libc-2.28.so [.] __nanosleep 0.00% sleep [unknown] [k] 0xffffffffaea00163 # # (Tip: To see callchains in a more compact form: perf report -g folded) # $ Signed-off-by:Alexey Budankov <alexey.budankov@linux.intel.com> Reviewed-by:
Jiri Olsa <jolsa@kernel.org> Tested-by:
Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com> Cc: Andi Kleen <ak@linux.intel.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/304b0a59-942c-3fe1-da02-aa749f87108b@linux.intel.comSigned-off-by:
-
Alexey Budankov authored
Implemented -z,--compression_level[=<n>] option that enables compression of mmaped kernel data buffers content in runtime during perf record mode collection. Default option value is 1 (fastest compression). Compression overhead has been measured for serial and AIO streaming when profiling matrix multiplication workload: ------------------------------------------------------------- | SERIAL | AIO-1 | ----------------------------------------------------------------| |-z | OVH(x) | ratio(x) size(MiB) | OVH(x) | ratio(x) size(MiB) | |---------------------------------------------------------------| | 0 | 1,00 | 1,000 179,424 | 1,00 | 1,000 187,527 | | 1 | 1,04 | 8,427 181,148 | 1,01 | 8,474 188,562 | | 2 | 1,07 | 8,055 186,953 | 1,03 | 7,912 191,773 | | 3 | 1,04 | 8,283 181,908 | 1,03 | 8,220 191,078 | | 5 | 1,09 | 8,101 187,705 | 1,05 | 7,780 190,065 | | 8 | 1,05 | 9,217 179,191 | 1,12 | 6,111 193,024 | ----------------------------------------------------------------- OVH = (Execution time with -z N) / (Execution time with -z 0) ratio - compression ratio size - number of bytes that was compressed size ~= trace size x ratio Committer notes: Testing it I noticed that it failed to disable build id processing when compression is enabled, and as we'd have to uncompress everything to look for the PERF_RECORD_{MMAP,SAMPLE,etc} to figure out which build ids to read from DSOs, we better disable build id processing when compression is enabled, logging with pr_debug() when doing so: Original patch: # perf record -z2 ^C[ perf record: Woken up 1 times to write data ] 0x1746e0 [0x76]: failed to process type: 81 [Invalid argument] [ perf record: Captured and wrote 1.568 MB perf.data, compressed (original 0.452 MB, ratio is 3.995) ] # After auto-disabling build id processing when compression is enabled: $ perf record -z2 sleep 1 [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.001 MB perf.data, compressed (original 0.001 MB, ratio is 2.292) ] $ perf record -v -z2 sleep 1 Compression enabled, disabling build id collection at the end of the session. <SNIP extra -v pr_debug() messages> [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.001 MB perf.data, compressed (original 0.001 MB, ratio is 2.305) ] $ Also, with parts of the patch originally after this one moved to just before this one we get: $ perf record -z2 sleep 1 [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.001 MB perf.data, compressed (original 0.001 MB, ratio is 2.371) ] $ perf report -D | grep COMPRESS 0 0x1b8 [0x155]: PERF_RECORD_COMPRESSED: unhandled! 0 0x30d [0x80]: PERF_RECORD_COMPRESSED: unhandled! COMPRESSED events: 2 COMPRESSED events: 0 $ I.e. when faced with PERF_RECORD_COMPRESSED that we still have no code to process, we just show it as not being handled, skip them and continue, while before we had: $ perf report -D | grep COMPRESS 0x1b8 [0x169]: failed to process type: 81 [Invalid argument] Error: failed to process sample 0 0x1b8 [0x169]: PERF_RECORD_COMPRESSED $ Signed-off-by: -
Alexey Budankov authored
Committer note: Split from a larger patch, this only dumps PERF_RECORD_COMPRESSED as unhandled, so that when we introduce the record part in the next patch, we don't see unhandled events when using 'perf record -D'. Changed it so that we dump the event if the handler is just a stub, i.e. for the case where we don't have ZSTD linked but we're processing a perf.data file generated by a tool with that linked. Also when failing to decompress we can't just dump the uncompressed event and return 0, we have to propagate the error. Signed-off-by:
-
Alexey Budankov authored
Compression is implemented using the functions from zstd.c. As the memory to operate on the compression uses mmap->aio.data[] buffers. If Zstd streaming compression API fails for some reason the data to be compressed are just copied into the memory buffers using plain memcpy(). Compressed trace frame consists of an array of PERF_RECORD_COMPRESSED records. Each element of the array is not longer that PERF_SAMPLE_MAX_SIZE and consists of perf_event_header followed by the compressed chunk that is decompressed on the loading stage. perf_mmap__aio_push() is replaced by perf_mmap__push() which is now used in the both serial and AIO streaming cases. perf_mmap__push() is extended with positive return values to signify absence of data ready for processing. Signed-off-by:
-
Alexey Budankov authored
Compression is implemented using the functions from zstd.c. As the memory to operate on the compression uses mmap->data buffer. If Zstd streaming compression API fails for some reason the data to be compressed are just copied into the memory buffers using plain memcpy(). Compressed trace frame consists of an array of PERF_RECORD_COMPRESSED records. Each element of the array is not longer that PERF_SAMPLE_MAX_SIZE and consists of perf_event_header followed by the compressed chunk that is decompressed on the loading stage. Comitter notes: Undo some unnecessary line breaks, remove some unnecessary () around zstd_data to then just get its address, and fix conflicts with BPF_PROG_INFO/BPF_BTF patchkits. Signed-off-by:
-
Alexey Budankov authored
Implemented functions are based on Zstd streaming compression API. The functions are used in runtime to compress data that come from mmaped kernel buffer. zstd_init(), zstd_fini() are used for initialization and finalization to allocate and deallocate internal zstd objects. zstd_compress_stream_to_records() is used to convert parts of mmaped kernel buffer into an array of PERF_RECORD_COMPRESSED records. Signed-off-by:
-
Alexey Budankov authored
Implemented mmap data buffer that is used as the memory to operate on when compressing data in case of serial trace streaming. Signed-off-by:
-
Alexey Budankov authored
Implemented PERF_RECORD_COMPRESSED event, related data types, header feature and functions to write, read and print feature attributes from the trace header section. comp_mmap_len preserves the size of mmaped kernel buffer that was used during collection. comp_mmap_len size is used on loading stage as the size of decomp buffer for decompression of COMPRESSED events content. Committer notes: Fixed up conflict with BPF_PROG_INFO and BTF_BTF header features. Signed-off-by:
-
Alexey Budankov authored
Define 'bytes_transferred' and 'bytes_compressed' metrics to calculate ratio in the end of the data collection: compression ratio = bytes_transferred / bytes_compressed The 'bytes_transferred' metric accumulates the amount of bytes that was extracted from the mmaped kernel buffers for compression, while 'bytes_compressed' accumulates the amount of bytes that was received after applying compression. Signed-off-by:
-
Arnaldo Carvalho de Melo authored
So that we can test the ifdef parts for this feature. Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com> Cc: Alexey Budankov <alexey.budankov@linux.intel.com> Cc: Andi Kleen <ak@linux.intel.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Peter Zijlstra <peterz@infradead.org> Link: https://lkml.kernel.org/n/tip-7o65mfl10wlvm8v3f0ombxd1@git.kernel.orgSigned-off-by:
-
Tzvetomir Stoyanov authored
Create man pages for trace sequences libtraceevent APIs: trace_seq_init(), trace_seq_destroy(), trace_seq_reset(), trace_seq_terminate(), trace_seq_putc(), trace_seq_puts(), trace_seq_printf(), trace_seq_vprintf(), trace_seq_do_fprintf(), trace_seq_do_printf() Signed-off-by:
Tzvetomir Stoyanov <tstoyanov@vmware.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Namhyung Kim <namhyung@kernel.org> Cc: linux-trace-devel@vger.kernel.org Link: http://lore.kernel.org/linux-trace-devel/20190503091119.23399-27-tstoyanov@vmware.com Link: http://lkml.kernel.org/r/20190510200110.462646052@goodmis.orgSigned-off-by:
Steven Rostedt (VMware) <rostedt@goodmis.org> Signed-off-by:
-
Tzvetomir Stoyanov authored
Create man pages for libtraceevent APIs: tep_data_type(), tep_data_pid(), tep_data_preempt_count(), tep_data_flags() Signed-off-by:
-
Tzvetomir Stoyanov authored
Create man page for tep_parse_header_page() libtraceevent API. Signed-off-by:
-
Tzvetomir Stoyanov authored
Create man pages for libtraceevent APIs: tep_parse_event(), tep_parse_format() Signed-off-by:
-
Tzvetomir Stoyanov authored
Added new man pages, describing libtraceevent event filter APIs: tep_filter_alloc() tep_filter_free() tep_filter_reset() tep_filter_add_filter_str() tep_filter_strerror() tep_event_filtered() tep_filter_remove_event() tep_filter_match() tep_filter_copy() tep_filter_compare() tep_filter_make_string() Signed-off-by:
-
Tzvetomir Stoyanov authored
Create man pages for libtraceevent APIs: tep_event_common_fields(), tep_event_fields() Signed-off-by:
-
Tzvetomir Stoyanov authored
Create man page for libtraceevent API tep_read_number_field(). Signed-off-by:
-
Tzvetomir Stoyanov authored
Create man pages for libtraceevent APIs: tep_print_field(), tep_print_fields(), tep_print_num_field(), tep_print_func_field() Signed-off-by:
-
Tzvetomir Stoyanov authored
Create man pages for libtraceevent APIs: tep_get_any_field_val(), tep_get_common_field_val(), tep_get_field_val(), tep_get_field_raw() Signed-off-by:
-
Tzvetomir Stoyanov authored
Create man pages for libtraceevent APIs: tep_find_common_field(), tep_find_field() tep_find_any_field() Signed-off-by:
-
Tzvetomir Stoyanov authored
Create man pages for libtraceevent APIs: tep_get_event(), tep_get_first_event(), tep_get_events_count() Signed-off-by:
-
Tzvetomir Stoyanov authored
Create man page for libtraceevent APIs: tep_list_events() tep_list_events_copy() Signed-off-by:
-
Tzvetomir Stoyanov authored
Create man pages for libtraceevent APIs: tep_find_event() tep_find_event_by_name() tep_find_event_by_record() Signed-off-by:
-
Tzvetomir Stoyanov authored
Create man page for tep_read_number() libtraceevent API. Signed-off-by:
-
Tzvetomir Stoyanov authored
Create man pages for libtraceevent APIs: tep_register_print_function() tep_unregister_print_function() Signed-off-by:
-
Tzvetomir Stoyanov authored
Added new man pages, describing function related libtraceevent APIs: tep_register_function(), tep_register_print_string(), tep_find_function(), tep_find_function_address(), tep_set_function_resolver(), tep_reset_function_resolver() Signed-off-by:
-
Tzvetomir Stoyanov authored
Create man pages for libtraceevent APIs: tep_register_event_handler() tep_unregister_event_handler() Signed-off-by:
-
Tzvetomir Stoyanov authored
Create man page for tep_strerror() libtraceevent API. Signed-off-by:
-
Tzvetomir Stoyanov authored
Create man pages for libtraceevent APIs: tep_get_page_size() tep_set_page_size() Signed-off-by:
-
Tzvetomir Stoyanov authored
Create man pages for libtraceevent APIs: tep_is_bigendian(), tep_is_local_bigendian(), tep_set_local_bigendian() Signed-off-by:
-
Tzvetomir Stoyanov authored
Create man pages for libtraceevent APIs: tep_is_file_bigendian(), tep_set_file_bigendian() Signed-off-by:
-
Tzvetomir Stoyanov authored
Create man pages for libtraceevent APIs: tep_get_cpus(), tep_set_cpus() Signed-off-by:
-
Tzvetomir Stoyanov authored
Create a man page for libtraceevent APIs: tep_get_header_page_size(), tep_get_header_timestamp_size(), tep_is_old_format() Signed-off-by:
-
Tzvetomir Stoyanov authored
Added 4 new man pages, describing libtraceevent APIs: tep_register_comm(), tep_override_comm(), tep_is_pid_registered(), tep_data_comm_from_pid(), tep_data_pid_from_comm(), tep_cmdline_pid(), tep_alloc(), tep_free(), tep_get_long_size(), tep_set_long_size(), tep_set_flag(), tep_clear_flag(), tep_test_flag() Signed-off-by:
-
Tzvetomir Stoyanov authored
Added support for man pages with multiple names, used to combine the description of several APIs into one page. Signed-off-by:
-
Tzvetomir Stoyanov authored
Initial support for libtraceevent man pages - Documentation directory, templates, configurations, Makefiles. The first man page is also part of the patch - summary of the library and all its APIs. Building of the documentation is integrated into the libtraceevent build process, new targets are added to its Makefile: make help make doc make doc-clean make doc-install make doc-uninstall Signed-off-by:
-
Donald Yandt authored
If fgets() fails due to any other error besides end-of-file, the version char array may not even be null-terminated. Signed-off-by:
Donald Yandt <donald.yandt@gmail.com> Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com> Cc: Avi Kivity <avi@scylladb.com> Cc: Jiri Olsa <jolsa@redhat.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Yanmin Zhang <yanmin_zhang@linux.intel.com> Fixes: a1645ce1 ("perf: 'perf kvm' tool for monitoring guest performance from host") Link: http://lkml.kernel.org/r/20190514110100.22019-1-donald.yandt@gmail.comSigned-off-by:
-
Kan Liang authored
Perf cannot parse UPI (Intel's "Ultra Path Interconnect" [1]) events. # perf stat -e UPI_DATA_BANDWIDTH_TX event syntax error: 'UPI_DATA_BANDWIDTH_TX' \___ parser error Run 'perf list' for a list of valid events The JSON lists call the box UPI LL, while perf calls it upi. Add conversion support to JSON to convert the unit properly. Committer notes: [1] https://en.wikipedia.org/wiki/Intel_Ultra_Path_Interconnect "The Intel Ultra Path Interconnect (UPI) is a point-to-point processor interconnect developed by Intel which replaced the Intel QuickPath Interconnect (QPI) in Xeon Skylake-SP platforms starting in 2017. UPI is a low-latency coherent interconnect for scalable multiprocessor systems with a shared address space. It uses a directory-based home snoop coherency protocol with a transfer speed of up to 10.4 GT/s. Supporting processors typically have two or three UPI links." Signed-off-by:Kan Liang <kan.liang@linux.intel.com> Cc: Andi Kleen <ak@linux.intel.com> Cc: Jiri Olsa <jolsa@kernel.org> Link: http://lkml.kernel.org/r/1557234991-130456-1-git-send-email-kan.liang@linux.intel.comSigned-off-by:
-
Adrian Hunter authored
With support for Python 2 or 3 and PySide 1 or 2 (Qt 4 or 5), it is useful to see what versions are in use. Add an 'About' dialog box that displays Python, PySide, Qt and database server (SQLite or PostgreSQL) version numbers. Committer testing: $ python ~acme/libexec/perf-core/scripts/python/exported-sql-viewer.py ~/c/adrian.hunter/simple-retpoline.db Then go to 'Help', then 'About', select all the lines with the mouse press 'Control+C', then, on the same terminal press control+shift+V which shows my current environment: Python version: 2.7.16 PySide version: 1 Qt version: 4.8.7 SQLite version: 3.26.0 Signed-off-by:
Adrian Hunter <adrian.hunter@intel.com> Tested-by:
-
Adrian Hunter authored
Add a context menu (right-click) that provides options for copying to clipboard, including, for trees, the ability to copy only the cell under the mouse pointer. Committer testing: $ python ~acme/libexec/perf-core/scripts/python/exported-sql-viewer.py ~/c/adrian.hunter/simple-retpoline.db Simply right click and pick "Copy selection", that at this point has just the first line, not expanded, then see what was copied by pressing shift+control+v on a terminal: Call Path,Object,Count,Time (ns),Time (%),Branch Count,Branch Count (%)
▶ simple-retpolin,,,,,, Ditto after expanding, i.e. the selection continues to be just one line: Call Path Object Count Time (ns) Time (%) Branch Count Branch Count (%) ▼ simple-retpolin Now select all the lines with the mouse and control+shift+v again: Call Path Object Count Time (ns) Time (%) Branch Count Branch Count (%) ▼ 14503:14503 ▼ _start ld-2.28.so 1 156267 100.0 10602 100.0▶ unknown unknown 1 2276 1.5 1 0.0▶ _dl_start ld-2.28.so 1 137047 87.7 10088 95.2▶ _dl_init ld-2.28.so 1 9142 5.9 326 3.1 ▼ _start simple-retpoline 1 7457 4.8 182 1.7▶ unknown unknown 1 805 10.8 1 0.5▶ __libc_start_main libc-2.28.so 1 6347 85.1 179 98.4 Signed-off-by:
-
