Commit Graph

20 Commits

Author SHA1 Message Date
Alberto Escolar Piedras
05f5390b92 twister tests: Use native_sim instead of native_posix
native_posix is deprecated. native_sim should be used in its place.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2024-08-16 09:21:08 +01:00
Anas Nashif
a9babb5a83 twister: tests: use platforms that do not depend on a HAL
Do not depend on platforms that need a HAL. This should speed things up
and should resolve issues where runner did not have enough space to deal
with all HALs.
t
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-08-06 08:47:01 -04:00
Jordan Yates
243eb36b18 everywhere: reindent .overlay files with tabs
`checkpatch.pl` requires that dts sources are indented with tabs,
fix all the spaces that slipped in while checkpatch wasn't watching.

Signed-off-by: Jordan Yates <jordan@embeint.com>
2024-06-26 15:59:44 +02:00
Kamil Paszkiet
85c6ce69ca scripts: tests: Blackbox test expansion - report summary
add test for report summary switch

Signed-off-by: Kamil Paszkiet <kamilx.paszkiet@intel.com>
2024-05-15 09:29:05 +02:00
Lukasz Mrugala
a294a9a9ab scripts: twister: Name prefix loading
Sped up Twister via name prefix loading.
If the only thing influencing the platform selection
is the platform filter (-p), then we only load *.yaml
files that start the same way as those selected platforms.
We split the platform name to the first '/' or '@'.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-05-07 09:38:00 +02:00
Patryk Kuniecki
d8b368a160 scripts: tests: twister_blackbox: Add test_ignore_platform_key
Add new tests to test_filter.py in blackbox twister testing

Signed-off-by: Patryk Kuniecki <patryk.kuniecki@intel.com>
2024-04-03 10:34:35 +02:00
Lukasz Mrugala
2f97795360 scripts: tests: Blackbox test expansion - platform
Adds tests related to the Twister's platform selection:
* -A, --board-root
* -K, --force-platform
* -p, --platform

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-03-31 17:37:09 -04:00
Kamil Paszkiet
717fccaa09 scripts: tests: Blackbox test expansion - device
Adds tests related to device flags:
--seed

Signed-off-by: Kamil Paszkiet <kamilx.paszkiet@intel.com>
2024-03-29 15:07:38 +00:00
Artur Wilczak
6d8132c445 scripts: tests: Blackbox test expension - disable
Add test related to disable-suite-name-check flag

This flag disabling extended test suite name verification at
the beginning of Ztest test. This option could be useful for tests
or platforms, which from some reasons cannot print early logs.

Add test related disable-warnings-as-errors

Do not treat warning conditions as errors.

Signed-off-by: Artur Wilczak <arturx.wilczak@intel.com>
2024-03-29 15:07:19 +00:00
Kamil Paszkiet
fff833eda4 scripts: tests: Blackbox test expansion - filter
Adds tests related to filter flags:
--arch
--vendor

Signed-off-by: Kamil Paszkiet <kamilx.paszkiet@intel.com>
2024-03-27 14:31:43 +00:00
Lukasz Mrugala
3fb11e260a scripts: tests: Blackbox test expansion - addon
Adds tests related to the addon flags:
*     --enable-ubsan
*     --enable-lsan
*     --enable-asan
*     --enable-valgrind
*     --allow-installed-plugin
*     --pytest-args
* -x, --extra-args
* extra test args for the binary

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-03-26 18:08:31 -05:00
Kamil Paszkiet
2a4332ba5f scripts: tests: Blackbox test expansion - error
Adds tests related to filter flags:
--overflow-as-errors

Signed-off-by: Kamil Paszkiet <kamilx.paszkiet@intel.com>
2024-03-26 11:10:02 -05:00
Alberto Escolar Piedras
a4b0e8a706 twister tests: Let's use native_sim instead of native_posix
native_posix is being replaced with native_sim, let's
use native_sim instead in twister tests.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2024-03-19 13:15:48 -05:00
Lukasz Mrugala
93a377e674 scripts: tests: Blackbox test expansion - outfile
Adds tests related to the changing of the output files:
* -c, --clobber-output
* -n, --no-clean
* -M, --runtime-artifact-cleanup
*     --short-build-path
*     --prep-artifacts-for-testing

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-03-05 15:56:04 +01:00
Lukasz Mrugala
a815f70177 scripts: tests: Blackbox test expansion - config
Adds tests related to the flags concerned with config files:
*     --alt-config-root

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-03-05 07:44:51 -05:00
Paszkiet Kamil
1799a2bf0c scripts: tests: twister_blackbox: Add test test_runner.py
- change name test_qemu.py to test_runner.py
- add test to test_runner.py:
  - build_only
  - runtest_only
  - ninja (dummy test)
  - dry_run (dummy test)
  - any_platform (dummy test)
  - cmake_only
  - pre_script (dummy test)
  - exclude_platform
  - device_flash_timeout (dummy test)
  - retry_failed
  - retry_interval
  - timeout_multiplier
  - quarantine_list
  - tag
  - only_failed
  - retry_build_errors

Made required changes to test_testplan.py tests

Signed-off-by: Paszkiet Kamil <kamilx.paszkiet@intel.com>
2024-01-29 17:43:59 -05:00
Lukasz Mrugala
c405d8d68f scripts: tests: Blackbox test expansion
Adds five new tests, covering six more flags:
* -s, --test
*     --sub-test
* -G, --integration
*     --test-config
*     --level
*     --filter

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-01-19 09:53:36 +01:00
Alberto Escolar Piedras
188cdd95c5 twister tests: Replace native_posix with native_sim
As native_sim is replacing native_posix overall and
becoming the default test platform.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-11-30 09:42:25 -05:00
Anas Nashif
719fee2bbc doc: ztest: adapt ztest docs
Remove remaining usage and documenation of dropped kconfig that is now
the default.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-10-20 15:04:29 +02:00
Lukasz Mrugala
4d467af7f4 scripts: tests: twister: Black box testing
In addition to the white-box testing and unit tests,
we would like to have some black-box testing for the
console commands of twister.

This serves as a Proof of Concept for future expansion.

Special dummy tests are made for blackbox testing.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2023-08-28 15:57:06 +02:00