Commit 595a56ac authored by Linus Torvalds's avatar Linus Torvalds

Merge tag 'linux-kselftest-kunit-5.8-rc1' of...

Merge tag 'linux-kselftest-kunit-5.8-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest

Pull Kunit updates from Shuah Khan:
 "This consists of:

   - Several config fragment fixes from Anders Roxell to improve test
     coverage.

   - Improvements to kunit run script to use defconfig as default and
     restructure the code for config/build/exec/parse from Vitor Massaru
     Iha and David Gow.

   - Miscellaneous documentation warn fix"

* tag 'linux-kselftest-kunit-5.8-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest:
  security: apparmor: default KUNIT_* fragments to KUNIT_ALL_TESTS
  fs: ext4: default KUNIT_* fragments to KUNIT_ALL_TESTS
  drivers: base: default KUNIT_* fragments to KUNIT_ALL_TESTS
  lib: Kconfig.debug: default KUNIT_* fragments to KUNIT_ALL_TESTS
  kunit: default KUNIT_* fragments to KUNIT_ALL_TESTS
  kunit: Kconfig: enable a KUNIT_ALL_TESTS fragment
  kunit: Fix TabError, remove defconfig code and handle when there is no kunitconfig
  kunit: use KUnit defconfig by default
  kunit: use --build_dir=.kunit as default
  Documentation: test.h - fix warnings
  kunit: kunit_tool: Separate out config/build/exec/parse
parents fc2fb38c 6d6861d4
...@@ -32,12 +32,14 @@ test targets as well. The ``.kunitconfig`` should also contain any other config ...@@ -32,12 +32,14 @@ test targets as well. The ``.kunitconfig`` should also contain any other config
options required by the tests. options required by the tests.
A good starting point for a ``.kunitconfig`` is the KUnit defconfig: A good starting point for a ``.kunitconfig`` is the KUnit defconfig:
.. code-block:: bash .. code-block:: bash
cd $PATH_TO_LINUX_REPO cd $PATH_TO_LINUX_REPO
cp arch/um/configs/kunit_defconfig .kunitconfig cp arch/um/configs/kunit_defconfig .kunitconfig
You can then add any other Kconfig options you wish, e.g.: You can then add any other Kconfig options you wish, e.g.:
.. code-block:: none .. code-block:: none
CONFIG_LIST_KUNIT_TEST=y CONFIG_LIST_KUNIT_TEST=y
...@@ -54,8 +56,8 @@ using. ...@@ -54,8 +56,8 @@ using.
other tools (such as make menuconfig) to adjust other config options. other tools (such as make menuconfig) to adjust other config options.
Running the tests Running the tests (KUnit Wrapper)
----------------- ---------------------------------
To make sure that everything is set up correctly, simply invoke the Python To make sure that everything is set up correctly, simply invoke the Python
wrapper from your kernel repo: wrapper from your kernel repo:
...@@ -105,8 +107,9 @@ have config options ending in ``_KUNIT_TEST``. ...@@ -105,8 +107,9 @@ have config options ending in ``_KUNIT_TEST``.
KUnit and KUnit tests can be compiled as modules: in this case the tests in a KUnit and KUnit tests can be compiled as modules: in this case the tests in a
module will be run when the module is loaded. module will be run when the module is loaded.
Running the tests
----------------- Running the tests (w/o KUnit Wrapper)
-------------------------------------
Build and run your kernel as usual. Test output will be written to the kernel Build and run your kernel as usual. Test output will be written to the kernel
log in `TAP <https://testanything.org/>`_ format. log in `TAP <https://testanything.org/>`_ format.
......
...@@ -595,7 +595,7 @@ able to run one test case per invocation. ...@@ -595,7 +595,7 @@ able to run one test case per invocation.
KUnit debugfs representation KUnit debugfs representation
============================ ============================
When kunit test suites are initialized, they create an associated directory When kunit test suites are initialized, they create an associated directory
in /sys/kernel/debug/kunit/<test-suite>. The directory contains one file in ``/sys/kernel/debug/kunit/<test-suite>``. The directory contains one file
- results: "cat results" displays results of each test case and the results - results: "cat results" displays results of each test case and the results
of the entire suite for the last test run. of the entire suite for the last test run.
...@@ -604,4 +604,4 @@ The debugfs representation is primarily of use when kunit test suites are ...@@ -604,4 +604,4 @@ The debugfs representation is primarily of use when kunit test suites are
run in a native environment, either as modules or builtin. Having a way run in a native environment, either as modules or builtin. Having a way
to display results like this is valuable as otherwise results can be to display results like this is valuable as otherwise results can be
intermixed with other events in dmesg output. The maximum size of each intermixed with other events in dmesg output. The maximum size of each
results file is KUNIT_LOG_SIZE bytes (defined in include/kunit/test.h). results file is KUNIT_LOG_SIZE bytes (defined in ``include/kunit/test.h``).
...@@ -149,8 +149,9 @@ config DEBUG_TEST_DRIVER_REMOVE ...@@ -149,8 +149,9 @@ config DEBUG_TEST_DRIVER_REMOVE
test this functionality. test this functionality.
config PM_QOS_KUNIT_TEST config PM_QOS_KUNIT_TEST
bool "KUnit Test for PM QoS features" bool "KUnit Test for PM QoS features" if !KUNIT_ALL_TESTS
depends on KUNIT=y depends on KUNIT=y
default KUNIT_ALL_TESTS
config HMEM_REPORTING config HMEM_REPORTING
bool bool
......
...@@ -9,5 +9,6 @@ config TEST_ASYNC_DRIVER_PROBE ...@@ -9,5 +9,6 @@ config TEST_ASYNC_DRIVER_PROBE
If unsure say N. If unsure say N.
config KUNIT_DRIVER_PE_TEST config KUNIT_DRIVER_PE_TEST
bool "KUnit Tests for property entry API" bool "KUnit Tests for property entry API" if !KUNIT_ALL_TESTS
depends on KUNIT=y depends on KUNIT=y
default KUNIT_ALL_TESTS
...@@ -102,9 +102,10 @@ config EXT4_DEBUG ...@@ -102,9 +102,10 @@ config EXT4_DEBUG
using dynamic debug control for mb_debug() / ext_debug() msgs. using dynamic debug control for mb_debug() / ext_debug() msgs.
config EXT4_KUNIT_TESTS config EXT4_KUNIT_TESTS
tristate "KUnit tests for ext4" tristate "KUnit tests for ext4" if !KUNIT_ALL_TESTS
select EXT4_FS select EXT4_FS
depends on KUNIT depends on KUNIT
default KUNIT_ALL_TESTS
help help
This builds the ext4 KUnit tests. This builds the ext4 KUnit tests.
......
...@@ -175,7 +175,7 @@ struct kunit_suite { ...@@ -175,7 +175,7 @@ struct kunit_suite {
void (*exit)(struct kunit *test); void (*exit)(struct kunit *test);
struct kunit_case *test_cases; struct kunit_case *test_cases;
/* private - internal use only */ /* private: internal use only */
struct dentry *debugfs; struct dentry *debugfs;
char *log; char *log;
}; };
...@@ -232,9 +232,9 @@ void __kunit_test_suites_exit(struct kunit_suite **suites); ...@@ -232,9 +232,9 @@ void __kunit_test_suites_exit(struct kunit_suite **suites);
* kunit_test_suites() - used to register one or more &struct kunit_suite * kunit_test_suites() - used to register one or more &struct kunit_suite
* with KUnit. * with KUnit.
* *
* @suites: a statically allocated list of &struct kunit_suite. * @suites_list...: a statically allocated list of &struct kunit_suite.
* *
* Registers @suites with the test framework. See &struct kunit_suite for * Registers @suites_list with the test framework. See &struct kunit_suite for
* more information. * more information.
* *
* When builtin, KUnit tests are all run as late_initcalls; this means * When builtin, KUnit tests are all run as late_initcalls; this means
...@@ -253,8 +253,8 @@ void __kunit_test_suites_exit(struct kunit_suite **suites); ...@@ -253,8 +253,8 @@ void __kunit_test_suites_exit(struct kunit_suite **suites);
* tests from the same place, and at the very least to do so after * tests from the same place, and at the very least to do so after
* everything else is definitely initialized. * everything else is definitely initialized.
*/ */
#define kunit_test_suites(...) \ #define kunit_test_suites(suites_list...) \
static struct kunit_suite *suites[] = { __VA_ARGS__, NULL}; \ static struct kunit_suite *suites[] = {suites_list, NULL}; \
static int kunit_test_suites_init(void) \ static int kunit_test_suites_init(void) \
{ \ { \
return __kunit_test_suites_init(suites); \ return __kunit_test_suites_init(suites); \
......
...@@ -2142,8 +2142,9 @@ config TEST_SYSCTL ...@@ -2142,8 +2142,9 @@ config TEST_SYSCTL
If unsure, say N. If unsure, say N.
config SYSCTL_KUNIT_TEST config SYSCTL_KUNIT_TEST
tristate "KUnit test for sysctl" tristate "KUnit test for sysctl" if !KUNIT_ALL_TESTS
depends on KUNIT depends on KUNIT
default KUNIT_ALL_TESTS
help help
This builds the proc sysctl unit test, which runs on boot. This builds the proc sysctl unit test, which runs on boot.
Tests the API contract and implementation correctness of sysctl. Tests the API contract and implementation correctness of sysctl.
...@@ -2153,8 +2154,9 @@ config SYSCTL_KUNIT_TEST ...@@ -2153,8 +2154,9 @@ config SYSCTL_KUNIT_TEST
If unsure, say N. If unsure, say N.
config LIST_KUNIT_TEST config LIST_KUNIT_TEST
tristate "KUnit Test for Kernel Linked-list structures" tristate "KUnit Test for Kernel Linked-list structures" if !KUNIT_ALL_TESTS
depends on KUNIT depends on KUNIT
default KUNIT_ALL_TESTS
help help
This builds the linked list KUnit test suite. This builds the linked list KUnit test suite.
It tests that the API and basic functionality of the list_head type It tests that the API and basic functionality of the list_head type
......
...@@ -15,7 +15,8 @@ menuconfig KUNIT ...@@ -15,7 +15,8 @@ menuconfig KUNIT
if KUNIT if KUNIT
config KUNIT_DEBUGFS config KUNIT_DEBUGFS
bool "KUnit - Enable /sys/kernel/debug/kunit debugfs representation" bool "KUnit - Enable /sys/kernel/debug/kunit debugfs representation" if !KUNIT_ALL_TESTS
default KUNIT_ALL_TESTS
help help
Enable debugfs representation for kunit. Currently this consists Enable debugfs representation for kunit. Currently this consists
of /sys/kernel/debug/kunit/<test_suite>/results files for each of /sys/kernel/debug/kunit/<test_suite>/results files for each
...@@ -23,7 +24,8 @@ config KUNIT_DEBUGFS ...@@ -23,7 +24,8 @@ config KUNIT_DEBUGFS
run that occurred. run that occurred.
config KUNIT_TEST config KUNIT_TEST
tristate "KUnit test for KUnit" tristate "KUnit test for KUnit" if !KUNIT_ALL_TESTS
default KUNIT_ALL_TESTS
help help
Enables the unit tests for the KUnit test framework. These tests test Enables the unit tests for the KUnit test framework. These tests test
the KUnit test framework itself; the tests are both written using the KUnit test framework itself; the tests are both written using
...@@ -32,7 +34,8 @@ config KUNIT_TEST ...@@ -32,7 +34,8 @@ config KUNIT_TEST
expected. expected.
config KUNIT_EXAMPLE_TEST config KUNIT_EXAMPLE_TEST
tristate "Example test for KUnit" tristate "Example test for KUnit" if !KUNIT_ALL_TESTS
default KUNIT_ALL_TESTS
help help
Enables an example unit test that illustrates some of the basic Enables an example unit test that illustrates some of the basic
features of KUnit. This test only exists to help new users understand features of KUnit. This test only exists to help new users understand
...@@ -41,4 +44,18 @@ config KUNIT_EXAMPLE_TEST ...@@ -41,4 +44,18 @@ config KUNIT_EXAMPLE_TEST
is intended for curious hackers who would like to understand how to is intended for curious hackers who would like to understand how to
use KUnit for kernel development. use KUnit for kernel development.
config KUNIT_ALL_TESTS
tristate "All KUnit tests with satisfied dependencies"
help
Enables all KUnit tests, if they can be enabled.
KUnit tests run during boot and output the results to the debug log
in TAP format (http://testanything.org/). Only useful for kernel devs
running the KUnit test harness, and not intended for inclusion into a
production build.
For more information on KUnit and unit tests in general please refer
to the KUnit documentation in Documentation/dev-tools/kunit/.
If unsure, say N.
endif # KUNIT endif # KUNIT
...@@ -70,8 +70,9 @@ config SECURITY_APPARMOR_DEBUG_MESSAGES ...@@ -70,8 +70,9 @@ config SECURITY_APPARMOR_DEBUG_MESSAGES
the kernel message buffer. the kernel message buffer.
config SECURITY_APPARMOR_KUNIT_TEST config SECURITY_APPARMOR_KUNIT_TEST
bool "Build KUnit tests for policy_unpack.c" bool "Build KUnit tests for policy_unpack.c" if !KUNIT_ALL_TESTS
depends on KUNIT=y && SECURITY_APPARMOR depends on KUNIT=y && SECURITY_APPARMOR
default KUNIT_ALL_TESTS
help help
This builds the AppArmor KUnit tests. This builds the AppArmor KUnit tests.
......
This diff is collapsed.
...@@ -239,6 +239,24 @@ class KUnitMainTest(unittest.TestCase): ...@@ -239,6 +239,24 @@ class KUnitMainTest(unittest.TestCase):
self.print_patch.stop() self.print_patch.stop()
pass pass
def test_config_passes_args_pass(self):
kunit.main(['config'], self.linux_source_mock)
assert self.linux_source_mock.build_reconfig.call_count == 1
assert self.linux_source_mock.run_kernel.call_count == 0
def test_build_passes_args_pass(self):
kunit.main(['build'], self.linux_source_mock)
assert self.linux_source_mock.build_reconfig.call_count == 0
self.linux_source_mock.build_um_kernel.assert_called_once_with(False, 8, '', None)
assert self.linux_source_mock.run_kernel.call_count == 0
def test_exec_passes_args_pass(self):
kunit.main(['exec'], self.linux_source_mock)
assert self.linux_source_mock.build_reconfig.call_count == 0
assert self.linux_source_mock.run_kernel.call_count == 1
self.linux_source_mock.run_kernel.assert_called_once_with(build_dir='', timeout=300)
self.print_mock.assert_any_call(StrContains('Testing complete.'))
def test_run_passes_args_pass(self): def test_run_passes_args_pass(self):
kunit.main(['run'], self.linux_source_mock) kunit.main(['run'], self.linux_source_mock)
assert self.linux_source_mock.build_reconfig.call_count == 1 assert self.linux_source_mock.build_reconfig.call_count == 1
...@@ -247,6 +265,13 @@ class KUnitMainTest(unittest.TestCase): ...@@ -247,6 +265,13 @@ class KUnitMainTest(unittest.TestCase):
build_dir='', timeout=300) build_dir='', timeout=300)
self.print_mock.assert_any_call(StrContains('Testing complete.')) self.print_mock.assert_any_call(StrContains('Testing complete.'))
def test_exec_passes_args_fail(self):
self.linux_source_mock.run_kernel = mock.Mock(return_value=[])
with self.assertRaises(SystemExit) as e:
kunit.main(['exec'], self.linux_source_mock)
assert type(e.exception) == SystemExit
assert e.exception.code == 1
def test_run_passes_args_fail(self): def test_run_passes_args_fail(self):
self.linux_source_mock.run_kernel = mock.Mock(return_value=[]) self.linux_source_mock.run_kernel = mock.Mock(return_value=[])
with self.assertRaises(SystemExit) as e: with self.assertRaises(SystemExit) as e:
...@@ -257,14 +282,28 @@ class KUnitMainTest(unittest.TestCase): ...@@ -257,14 +282,28 @@ class KUnitMainTest(unittest.TestCase):
assert self.linux_source_mock.run_kernel.call_count == 1 assert self.linux_source_mock.run_kernel.call_count == 1
self.print_mock.assert_any_call(StrContains(' 0 tests run')) self.print_mock.assert_any_call(StrContains(' 0 tests run'))
def test_exec_raw_output(self):
self.linux_source_mock.run_kernel = mock.Mock(return_value=[])
kunit.main(['exec', '--raw_output'], self.linux_source_mock)
assert self.linux_source_mock.run_kernel.call_count == 1
for kall in self.print_mock.call_args_list:
assert kall != mock.call(StrContains('Testing complete.'))
assert kall != mock.call(StrContains(' 0 tests run'))
def test_run_raw_output(self): def test_run_raw_output(self):
self.linux_source_mock.run_kernel = mock.Mock(return_value=[]) self.linux_source_mock.run_kernel = mock.Mock(return_value=[])
with self.assertRaises(SystemExit) as e:
kunit.main(['run', '--raw_output'], self.linux_source_mock) kunit.main(['run', '--raw_output'], self.linux_source_mock)
assert type(e.exception) == SystemExit
assert e.exception.code == 1
assert self.linux_source_mock.build_reconfig.call_count == 1 assert self.linux_source_mock.build_reconfig.call_count == 1
assert self.linux_source_mock.run_kernel.call_count == 1 assert self.linux_source_mock.run_kernel.call_count == 1
for kall in self.print_mock.call_args_list:
assert kall != mock.call(StrContains('Testing complete.'))
assert kall != mock.call(StrContains(' 0 tests run'))
def test_exec_timeout(self):
timeout = 3453
kunit.main(['exec', '--timeout', str(timeout)], self.linux_source_mock)
self.linux_source_mock.run_kernel.assert_called_once_with(build_dir='', timeout=timeout)
self.print_mock.assert_any_call(StrContains('Testing complete.'))
def test_run_timeout(self): def test_run_timeout(self):
timeout = 3453 timeout = 3453
...@@ -282,5 +321,21 @@ class KUnitMainTest(unittest.TestCase): ...@@ -282,5 +321,21 @@ class KUnitMainTest(unittest.TestCase):
build_dir=build_dir, timeout=300) build_dir=build_dir, timeout=300)
self.print_mock.assert_any_call(StrContains('Testing complete.')) self.print_mock.assert_any_call(StrContains('Testing complete.'))
def test_config_builddir(self):
build_dir = '.kunit'
kunit.main(['config', '--build_dir', build_dir], self.linux_source_mock)
assert self.linux_source_mock.build_reconfig.call_count == 1
def test_build_builddir(self):
build_dir = '.kunit'
kunit.main(['build', '--build_dir', build_dir], self.linux_source_mock)
self.linux_source_mock.build_um_kernel.assert_called_once_with(False, 8, build_dir, None)
def test_exec_builddir(self):
build_dir = '.kunit'
kunit.main(['exec', '--build_dir', build_dir], self.linux_source_mock)
self.linux_source_mock.run_kernel.assert_called_once_with(build_dir=build_dir, timeout=300)
self.print_mock.assert_any_call(StrContains('Testing complete.'))
if __name__ == '__main__': if __name__ == '__main__':
unittest.main() unittest.main()
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment