bartoszek/zephyr
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
a86ad87d84
zephyr/scripts/tests/twister
History
Anas Nashif 14d88f8425 twister: add support for levels and test configuration 
Add support test levels and the ability to assign a specific test to one
or more levels. Using command line options of twister it is then possible
to select a level and just execute the tests included in this level.

Additionally, a test configuration allows definiing level
dependencies and additional inclusion of tests into a specific level if
the test itself does not have this information already.

In the configuration file you can include complete components using
regular expressions and you can specify which test level to import from
the same file, making management of levels easier.

To help with testing outside of upstream CI infrastructure, additional
options are available in the configuration file, which can be hosted
locally. As of now, those options are available:

- Ability to ignore default platforms as defined in board definitions
  (Those are mostly emulation platforms used to run tests in upstream
  CI)
- Option to specify your own list of default platforms overriding what
  upstream defines.
- Ability to override build_onl_all options used in some testscases.
  This will treat tests or sample as any other just build for default
  platforms you specify in the configuation file or on the command line.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-03-07 15:49:16 +01:00
..
test_data twister: add support for levels and test configuration 2023-03-07 15:49:16 +01:00
conftest.py twister: add support for levels and test configuration 2023-03-07 15:49:16 +01:00
README.md twister: fixups to various files 2022-07-05 12:06:27 -04:00
test_runner.py twister: Fields for Kconfig and DT overlay files in testcase.yaml 2023-03-01 16:52:01 -05:00
test_testinstance.py twister: cmake: Add prefiltration based on cmake package helper script 2023-03-02 15:38:08 -05:00
test_testplan_class.py twister: add support for levels and test configuration 2023-03-07 15:49:16 +01:00
test_testsuite.py Revert "twister: tests: Add unit tests for names of external and module tests" 2023-01-24 20:21:57 -05:00
test_twister.py twister: Fields for Kconfig and DT overlay files in testcase.yaml 2023-03-01 16:52:01 -05:00

README.md

Twister Testing

Running the tests require the environment variable ZEPHYR_BASE to be set.

Twister Testsuite are located in $ZEPHYR_BASE/scripts/tests directory with all the data files in $ZEPHYR_BASE/scripts/test_data directory.

Dependencies

Install all the dependencies using

pip install -r $ZEPHYR_BASE/scripts/requirements-build-test.txt

Executing testsuite

The testcases can be executed from the root directory using

pytest $ZEPHYR_BASE/scripts/tests/twister

Twister Coverage

The coverage for all the tests can be run using the command below. This will collect all the tests available.

coverage run -m pytest $ZEPHYR_BASE/scripts/tests/twister/

Then we can generate the coverage report for just twister script using

coverage report -m $ZEPHYR_BASE/scripts/pylib/twister/

To generate the coverage report for twister script use below command

coverage report -m $ZEPHYR_BASE/scripts/twister

The html coverage report for twister can be generated using

coverage html twister

If needed,the full coverage html report can be generated in every run of "pytest" in the tests directory using configuration file (setup.cfg).

Organization of tests

  • conftest.py: Contains common fixtures for use in testing the twister tool.
  • test_twister.py : Contains basic testcases for environment variables, verifying testcase & platform schema's.
  • test_testsuite_class.py : Contains testcases for Testsuite class (except reporting functionality) in twisterlib.py.
  • test_testinstance.py : Contains testcases for Testinstance and Testcase class.
  • test_reporting_testsuite.py : Contains testcases for reporting functionality of Testsuite class of twister.