Commit 7473ee56 authored by Claire Jensen's avatar Claire Jensen Committed by Arnaldo Carvalho de Melo
Browse files

perf test: Add checking for perf stat CSV output. 



Counts expected fields for various commands. No testing added for
summary mode since it is broken.

An example of the summary output is:

         summary,263831,,instructions:u,1435072,100.0,0.46,insn per cycle
,,,,,1.37,stalled cycles per insn

This should be:

         summary,263831,,instructions:u,1435072,100.0,0.46,insn per cycle
         summary,,,,,,1.37,stalled cycles per insn

The output has 7 fields when it should have 8. Additionally, the newline
spacing is wrong, so it was excluded from testing until a fix is made.

Committer testing:

  $ perf test "perf stat CSV output"
   88: perf stat CSV output linter                                     : Ok
  $

  $ perf test -v "perf stat CSV output"
  Couldn't bump rlimit(MEMLOCK), failures may take place when creating BPF maps, etc
   88: perf stat CSV output linter                                     :
  --- start ---
  test child forked, pid 2622839
  Checking CSV output: no args [Success]
  Checking CSV output: system wide [Skip] paranoid and not root
  Checking CSV output: system wide [Skip] paranoid and not root
  Checking CSV output: interval [Success]
  Checking CSV output: event [Success]
  Checking CSV output: per core [Skip] paranoid and not root
  Checking CSV output: per thread [Skip] paranoid and not root
  Checking CSV output: per die [Skip] paranoid and not root
  Checking CSV output: per node [Skip] paranoid and not root
  Checking CSV output: per socket [Skip] paranoid and not root
  test child finished with 0
  ---- end ----
  perf stat CSV output linter: Ok
  $

I did a s/parnoia/paranoid/g on the [Skip] lines.

Signed-off-by: default avatarClaire Jensen <cjense@google.com>
Tested-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Alyssa Ross <hi@alyssa.is>
Cc: Claire Jensen <clairej735@gmail.com>
Cc: Florian Fischer <florian.fischer@muhq.space>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: James Clark <james.clark@arm.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Like Xu <likexu@tencent.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sandipan Das <sandipan.das@amd.com>
Cc: Stephane Eranian <eranian@google.com>
Cc: Xing Zhengjun <zhengjun.xing@linux.intel.com>
Link: https://lore.kernel.org/r/20220525053814.3265216-2-irogers@google.com


Signed-off-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
parent a41e24f6

tools/perf/tests/shell/lib/perf_csv_output_lint.py

0 → 100644
+48 −0
Original line number Diff line number Diff line 
#!/usr/bin/python

# SPDX-License-Identifier: GPL-2.0



import argparse

import sys



# Basic sanity check of perf CSV output as specified in the man page.

# Currently just checks the number of fields per line in output.



ap = argparse.ArgumentParser()

ap.add_argument('--no-args', action='store_true')

ap.add_argument('--interval', action='store_true')

ap.add_argument('--system-wide-no-aggr', action='store_true')

ap.add_argument('--system-wide', action='store_true')

ap.add_argument('--event', action='store_true')

ap.add_argument('--per-core', action='store_true')

ap.add_argument('--per-thread', action='store_true')

ap.add_argument('--per-die', action='store_true')

ap.add_argument('--per-node', action='store_true')

ap.add_argument('--per-socket', action='store_true')

ap.add_argument('--separator', default=',', nargs='?')

args = ap.parse_args()



Lines = sys.stdin.readlines()



def check_csv_output(exp):

  for line in Lines:

    if 'failed' not in line:

      count = line.count(args.separator)

      if count != exp:

        sys.stdout.write(''.join(Lines))

        raise RuntimeError(f'wrong number of fields. expected {exp} in {line}')



try:

  if args.no_args or args.system_wide or args.event:

    expected_items = 6

  elif args.interval or args.per_thread or args.system_wide_no_aggr:

    expected_items = 7

  elif args.per_core or args.per_socket or args.per_node or args.per_die:

    expected_items = 8

  else:

    ap.print_help()

    raise RuntimeError('No checking option specified')

  check_csv_output(expected_items)



except:

  sys.stdout.write('Test failed for input: ' + ''.join(Lines))

  raise

tools/perf/tests/shell/stat+csv_output.sh

0 → 100755
+147 −0
Original line number Diff line number Diff line 
#!/bin/bash

# perf stat CSV output linter

# SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)

# Tests various perf stat CSV output commands for the

# correct number of fields and the CSV separator set to ','.



set -e



pythonchecker=$(dirname $0)/lib/perf_csv_output_lint.py

if [ "x$PYTHON" == "x" ]

then

	if which python3 > /dev/null

	then

		PYTHON=python3

	elif which python > /dev/null

	then

		PYTHON=python

	else

		echo Skipping test, python not detected please set environment variable PYTHON.

		exit 2

	fi

fi



# Return true if perf_event_paranoid is > $1 and not running as root.

function ParanoidAndNotRoot()

{

	 [ $(id -u) != 0 ] && [ $(cat /proc/sys/kernel/perf_event_paranoid) -gt $1 ]

}



check_no_args()

{

	echo -n "Checking CSV output: no args "

	perf stat -x, true 2>&1 | $PYTHON $pythonchecker --no-args

	echo "[Success]"

}



check_system_wide()

{

	echo -n "Checking CSV output: system wide "

	if ParanoidAndNotRoot 0

	then

		echo "[Skip] paranoid and not root"

		return

	fi

	perf stat -x, -a true 2>&1 | $PYTHON $pythonchecker --system-wide

	echo "[Success]"

}



check_system_wide_no_aggr()

{

	echo -n "Checking CSV output: system wide "

	if ParanoidAndNotRoot 0

	then

		echo "[Skip] paranoid and not root"

		return

	fi

	echo -n "Checking CSV output: system wide no aggregation "

	perf stat -x, -A -a --no-merge true 2>&1 | $PYTHON $pythonchecker --system-wide-no-aggr

	echo "[Success]"

}



check_interval()

{

	echo -n "Checking CSV output: interval "

	perf stat -x, -I 1000 true 2>&1 | $PYTHON $pythonchecker --interval

	echo "[Success]"

}





check_event()

{

	echo -n "Checking CSV output: event "

	perf stat -x, -e cpu-clock true 2>&1 | $PYTHON $pythonchecker --event

	echo "[Success]"

}



check_per_core()

{

	echo -n "Checking CSV output: per core "

	if ParanoidAndNotRoot 0

	then

		echo "[Skip] paranoid and not root"

		return

	fi

	perf stat -x, --per-core -a true 2>&1 | $PYTHON $pythonchecker --per-core

	echo "[Success]"

}



check_per_thread()

{

	echo -n "Checking CSV output: per thread "

	if ParanoidAndNotRoot 0

	then

		echo "[Skip] paranoid and not root"

		return

	fi

	perf stat -x, --per-thread -a true 2>&1 | $PYTHON $pythonchecker --per-thread

	echo "[Success]"

}



check_per_die()

{

	echo -n "Checking CSV output: per die "

	if ParanoidAndNotRoot 0

	then

		echo "[Skip] paranoid and not root"

		return

	fi

	perf stat -x, --per-die -a true 2>&1 | $PYTHON $pythonchecker --per-die

	echo "[Success]"

}



check_per_node()

{

	echo -n "Checking CSV output: per node "

	if ParanoidAndNotRoot 0

	then

		echo "[Skip] paranoid and not root"

		return

	fi

	perf stat -x, --per-node -a true 2>&1 | $PYTHON $pythonchecker --per-node

	echo "[Success]"

}



check_per_socket()

{

	echo -n "Checking CSV output: per socket "

	if ParanoidAndNotRoot 0

	then

		echo "[Skip] paranoid and not root"

		return

	fi

	perf stat -x, --per-socket -a true 2>&1 | $PYTHON $pythonchecker --per-socket

	echo "[Success]"

}



check_no_args

check_system_wide

check_system_wide_no_aggr

check_interval

check_event

check_per_core

check_per_thread

check_per_die

check_per_node

check_per_socket

exit 0