zephyr/scripts/tests/twister_blackbox
Dmitrii Golovanov f93f82f160 twister: coverage: Data collection and reporting per-test instance
With this change, the coverage data (GCOV dump) is extracted from the
test log files after each of the test instance parallel execution,
instead of post processing all the logs at the end of the Twister run.

The new `--coverage-per-instance` mode extends Twister coverage operations
to report coverage statistics on each test instance execution individually
in addition to the default reporting mode which aggregates data to one
report with all the test instances in the current scope of the Twister run.
The split mode allows to identify precisely what amount of code coverage
each test suite provides and to analyze its contribution to the overall
test plan's coverage. Each test configuration's output directory will have
its own coverage report and data files, so the overall disk space and
the total execution time increase.

Another new `--disable-coverage-aggregation` option allows to execute
only the `--coverage-per-instance` mode when the aggregate coverage
report for the whole Twister run scope is not needed.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2025-01-30 18:29:08 +01:00
..
test_data twister: ztest: short test case names on --no-detailed-test-id 2024-12-16 08:35:34 +01:00
conftest.py
pytest.ini
test_addon.py twister: test udpates for new board handling 2024-10-22 18:31:09 -04:00
test_config.py twister: ztest: harness: Fix missed TestCase statuses 2024-11-22 08:26:59 -05:00
test_coverage.py twister: coverage: Data collection and reporting per-test instance 2025-01-30 18:29:08 +01:00
test_device.py twister: support testing multiple toolchain variants 2025-01-08 12:58:59 +01:00
test_disable.py scripts: twister: decouple debug and verbosity 2024-09-20 11:07:48 +02:00
test_error.py twister: support testing multiple toolchain variants 2025-01-08 12:58:59 +01:00
test_filter.py twister: debug: change skipped -> filtered 2024-12-13 09:12:59 +01:00
test_footprint.py scripts: twister: Fix NOTRUN in test_only 2024-10-31 14:18:17 -05:00
test_hardwaremap.py
test_outfile.py twister: support testing multiple toolchain variants 2025-01-08 12:58:59 +01:00
test_output.py twister: support testing multiple toolchain variants 2025-01-08 12:58:59 +01:00
test_platform.py twister: fix Ztest C++ test names extraction from ELF 2024-11-22 08:26:59 -05:00
test_printouts.py twister: support testing multiple toolchain variants 2025-01-08 12:58:59 +01:00
test_quarantine.py twister: debug: change skipped -> filtered 2024-12-13 09:12:59 +01:00
test_report.py twister: do not create overlays/run_id for filtered cases 2024-12-04 02:02:52 +01:00
test_runner.py twister: support testing multiple toolchain variants 2025-01-08 12:58:59 +01:00
test_shuffle.py twister: fix Ztest C++ test names extraction from ELF 2024-11-22 08:26:59 -05:00
test_testlist.py twister: ztest: harness: Fix missed TestCase statuses 2024-11-22 08:26:59 -05:00
test_testplan.py twister: ztest: harness: Fix missed TestCase statuses 2024-11-22 08:26:59 -05:00
test_tooling.py scripts: twister: Unify Twister Statuses 2024-08-12 19:23:07 -04:00