f93f82f160
With this change, the coverage data (GCOV dump) is extracted from the test log files after each of the test instance parallel execution, instead of post processing all the logs at the end of the Twister run. The new `--coverage-per-instance` mode extends Twister coverage operations to report coverage statistics on each test instance execution individually in addition to the default reporting mode which aggregates data to one report with all the test instances in the current scope of the Twister run. The split mode allows to identify precisely what amount of code coverage each test suite provides and to analyze its contribution to the overall test plan's coverage. Each test configuration's output directory will have its own coverage report and data files, so the overall disk space and the total execution time increase. Another new `--disable-coverage-aggregation` option allows to execute only the `--coverage-per-instance` mode when the aggregate coverage report for the whole Twister run scope is not needed. Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com> |
||
|---|---|---|
| .. | ||
| pytest_integration | ||
| test_data | ||
| conftest.py | ||
| README.md | ||
| test_cmakecache.py | ||
| test_config_parser.py | ||
| test_environment.py | ||
| test_errors.py | ||
| test_handlers.py | ||
| test_hardwaremap.py | ||
| test_harness.py | ||
| test_jobserver.py | ||
| test_log_helper.py | ||
| test_mixins.py | ||
| test_platform.py | ||
| test_quarantine.py | ||
| test_runner.py | ||
| test_scl.py | ||
| test_testinstance.py | ||
| test_testplan.py | ||
| test_testsuite.py | ||
| test_twister.py | ||
Twister Testing
Running the tests require the environment variable ZEPHYR_BASE to be set.
Twister Testsuite are located in $ZEPHYR_BASE/scripts/tests directory with all the data files in $ZEPHYR_BASE/scripts/test_data directory.
Dependencies
Install all the dependencies using
pip install -r $ZEPHYR_BASE/scripts/requirements-build-test.txt
Executing testsuite
The testcases can be executed from the root directory using
pytest $ZEPHYR_BASE/scripts/tests/twister
Twister Coverage
The coverage for all the tests can be run using the command below. This will collect all the tests available.
coverage run -m pytest $ZEPHYR_BASE/scripts/tests/twister/
Then we can generate the coverage report for just twister script using
coverage report -m $ZEPHYR_BASE/scripts/pylib/twister/
To generate the coverage report for twister script use below command
coverage report -m $ZEPHYR_BASE/scripts/twister
The html coverage report for twister can be generated using
coverage html twister
If needed,the full coverage html report can be generated in every run of "pytest" in the tests directory using configuration file (setup.cfg).
Organization of tests
- conftest.py: Contains common fixtures for use in testing the twister tool.
- test_twister.py : Contains basic testcases for environment variables, verifying testcase & platform schema's.
- test_testsuite_class.py : Contains testcases for Testsuite class (except reporting functionality) in twisterlib.py.
- test_testinstance.py : Contains testcases for Testinstance and Testcase class.
- test_reporting_testsuite.py : Contains testcases for reporting functionality of Testsuite class of twister.