Commit e8b86f03 authored by Veronika Molnarova's avatar Veronika Molnarova Committed by Namhyung Kim

perf test stat_bpf_counter.sh: Stabilize the test results

The test has been failing for some time when two separate runs of
perf benchmarks are recorded for cycles events and their counts are
compared, while once the recording was done with option --bpf-counters
and once without it. It is expected that the count of the samples
should be within a certain range, firstly the difference was set to be
within 10%, which was then later raised to 20%. However, the test case
keeps failing on certain architectures as recording the provided
benchmark can produce completely different counts based on the
current load of the system.

Sampling two separate runs on intel-eaglestream-spr-13 of "perf stat
--no-big-num -e cycles -- perf bench sched messaging -g 1 -l 100 -t":

 Performance counter stats for 'perf bench sched messaging -g 1 -l 100 -t':

         396782898      cycles

       0.010051983 seconds time elapsed

       0.008664000 seconds user
       0.097058000 seconds sys

 Performance counter stats for 'perf bench sched messaging -g 1 -l 100 -t':

        1431133032      cycles

       0.021803714 seconds time elapsed

       0.023377000 seconds user
       0.349918000 seconds sys

, which is ranging from 400mil to 1400mil samples.

Instead of recording the cycles use instructions event, which provides
more stable values. At the same time change the tested workload to one
of the provided testing workloads by perf that is not based on a
scheduler, which can provide another dependency on the current load.

Sampling instructions event with the new workload provide much more
stable results on intel-eaglestream-spr-13 of "perf stat --no-big-num
-e instructions -- perf test -w brstack":

 Performance counter stats for 'perf test -w brstack':

          64584494      instructions

       0.009173945 seconds time elapsed

       0.007262000 seconds user
       0.002071000 seconds sys

 Performance counter stats for 'perf test -w brstack':

          64672669      instructions

       0.008888135 seconds time elapsed

       0.005018000 seconds user
       0.004018000 seconds sys
Signed-off-by: default avatarVeronika Molnarova <vmolnaro@redhat.com>
Acked-by: default avatarNamhyung Kim <namhyung@kernel.org>
Cc: mpetlan@redhat.com
Signed-off-by: default avatarNamhyung Kim <namhyung@kernel.org>
Link: https://lore.kernel.org/r/20240625092001.10909-1-vmolnaro@redhat.com
parent e4b19e2c
......@@ -4,7 +4,7 @@
set -e
workload="perf bench sched messaging -g 1 -l 100 -t"
workload="perf test -w brstack"
# check whether $2 is within +/- 20% of $1
compare_number()
......@@ -25,15 +25,15 @@ compare_number()
check_counts()
{
base_cycles=$1
bpf_cycles=$2
base_instructions=$1
bpf_instructions=$2
if [ "$base_cycles" = "<not" ]; then
echo "Skipping: cycles event not counted"
if [ "$base_instructions" = "<not" ]; then
echo "Skipping: instructions event not counted"
exit 2
fi
if [ "$bpf_cycles" = "<not" ]; then
echo "Failed: cycles not counted with --bpf-counters"
if [ "$bpf_instructions" = "<not" ]; then
echo "Failed: instructions not counted with --bpf-counters"
exit 1
fi
}
......@@ -41,29 +41,29 @@ check_counts()
test_bpf_counters()
{
printf "Testing --bpf-counters "
base_cycles=$(perf stat --no-big-num -e cycles -- $workload 2>&1 | awk '/cycles/ {print $1}')
bpf_cycles=$(perf stat --no-big-num --bpf-counters -e cycles -- $workload 2>&1 | awk '/cycles/ {print $1}')
check_counts $base_cycles $bpf_cycles
compare_number $base_cycles $bpf_cycles
base_instructions=$(perf stat --no-big-num -e instructions -- $workload 2>&1 | awk '/instructions/ {print $1}')
bpf_instructions=$(perf stat --no-big-num --bpf-counters -e instructions -- $workload 2>&1 | awk '/instructions/ {print $1}')
check_counts $base_instructions $bpf_instructions
compare_number $base_instructions $bpf_instructions
echo "[Success]"
}
test_bpf_modifier()
{
printf "Testing bpf event modifier "
stat_output=$(perf stat --no-big-num -e cycles/name=base_cycles/,cycles/name=bpf_cycles/b -- $workload 2>&1)
base_cycles=$(echo "$stat_output"| awk '/base_cycles/ {print $1}')
bpf_cycles=$(echo "$stat_output"| awk '/bpf_cycles/ {print $1}')
check_counts $base_cycles $bpf_cycles
compare_number $base_cycles $bpf_cycles
stat_output=$(perf stat --no-big-num -e instructions/name=base_instructions/,instructions/name=bpf_instructions/b -- $workload 2>&1)
base_instructions=$(echo "$stat_output"| awk '/base_instructions/ {print $1}')
bpf_instructions=$(echo "$stat_output"| awk '/bpf_instructions/ {print $1}')
check_counts $base_instructions $bpf_instructions
compare_number $base_instructions $bpf_instructions
echo "[Success]"
}
# skip if --bpf-counters is not supported
if ! perf stat -e cycles --bpf-counters true > /dev/null 2>&1; then
if ! perf stat -e instructions --bpf-counters true > /dev/null 2>&1; then
if [ "$1" = "-v" ]; then
echo "Skipping: --bpf-counters not supported"
perf --no-pager stat -e cycles --bpf-counters true || true
perf --no-pager stat -e instructions --bpf-counters true || true
fi
exit 2
fi
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment