Add the missing #ifndef CONFIG_ZTEST_SHUFFLE in ztest.c file.
This fixes multiple definitions of NUM_ITER_PER_SUITE
and NUM_ITER_PER_TEST.
Signed-off-by: Arkadiusz Cholewinski <arkadiuszx.cholewinski@intel.com>
Expanding the ZTEST shell to include a command that
accepts additional parameters, such as ztest run_testcase
<suite:test> -p any_value.
Adding the new ZTEST_P macro, to create the
parameterized test.
Signed-off-by: Arkadiusz Cholewinski <arkadiuszx.cholewinski@intel.com>
Expand ztest_shell subcommands `run-testcase`
and `run-testsuite` to include additional parameter `-r`
repeat-iter. This allows to run a particular testcase or
testsuite several times.
Signed-off-by: Arkadiusz Cholewinski <arkadiuszx.cholewinski@intel.com>
Replace `shell_fprintf(sh, SHELL_ERROR, ...)` with
`shell_error(sh, ...)` since it brings the same underlying
action with less typing.
Note: `shell_error` already concatenates `\n` to format strings,
so we remove `\n` from the original code.
Signed-off-by: Pisit Sawangvonganan <pisit@ndrsolution.com>
Utilize a code spell-checking tool to scan for and correct spelling errors
in `subsys/testsuite/ztest/src/ztest.c` file.
Signed-off-by: Pisit Sawangvonganan <pisit@ndrsolution.com>
Undefine _POSIX_C_SOURCE before defining it to avoid double
definition
The previous version, changed in this PR
https://github.com/zephyrproject-rtos/zephyr/pull/70363, did the undef.
Make sure to do the same for all files in the current dir.
Signed-off-by: Dawid Niedzwiecki <dawidn@google.com>
The testsuite subsystem provides the macro PRINT_DATA() which can be
overridden using CONFIG_ZTEST_TC_UTIL_USER_OVERRIDE. Switch all calls of
PRINT() to PRINT_DATA withing the testsuite subsystem.
Signed-off-by: Keith Short <keithshort@google.com>
Fix compilation error implicit declaration of function 'free'
in case of `CONFIG_NEWLIB_LIBC`.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Many ztest macros were not indexed correctly as part of the ztest_test
group, for example ZTEST_SUITE was completely missing in the
documentation.
In addition, document a few macros that did not have any documentation.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add an option to reset gcov counters before running tests.
This ensures that only code lines triggered by test itself are counted in
the coverage report and all the board initialization code and pre-test
bootstrap is not counted. This is useful when, for example, you are
testing code that is also executed during bootup
Test Plan:
west build -p always -b qemu_x86_64 tests/ztest/base/ -- \
-DCONFIG_COVERAGE=y -DCONFIG_COVERAGE_GCOV=y -DCONFIG_COVERAGE_DUMP=y \
-DCONFIG_ZTEST_COVERAGE_RESET_BEFORE_TESTS=y
ninja -Cbuild run | tee log.log
Signed-off-by: Roman Studenikin <srv@meta.com>
Make `struct arch_esf` compulsory for all architectures by
declaring it in the `arch_interface.h` header.
After this commit, the named struct `z_arch_esf_t` is only used
internally to generate offsets, and is slated to be removed
from the `arch_interface.h` header in the future.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Namespaced the generated headers with `zephyr` to prevent
potential conflict with other headers.
Introduce a temporary Kconfig `LEGACY_GENERATED_INCLUDE_PATH`
that is enabled by default. This allows the developers to
continue the use of the old include paths for the time being
until it is deprecated and eventually removed. The Kconfig will
generate a build-time warning message, similar to the
`CONFIG_TIMER_RANDOM_GENERATOR`.
Updated the includes path of in-tree sources accordingly.
Most of the changes here are scripted, check the PR for more
info.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
The `cpu.h` is meant to be a hack around Zephyr's dependencies
for unit-test, relocate it to a folder that's included only by
unit-test.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
When z_test_1cpu_start() is called to ensure that only a single CPU
on an SMP system is available for use in a test, this commit will
ensure that that CPU is the primary CPU--CPU0. This is done because
some timer drivers only have the timer interrupt processed by one CPU.
A bit of a song and dance is performed to achieve this without enabling
the CPU mask/affinity pinning API. If the cpuhold thread is found to
be executing on CPU0, then a new copy of cpuhold thread is created. Once
the new copy is executing (incidentally guaranteed to be on another CPU)
then it informs the original copy and busy waits until it the original
copy is switched out of CPU0. At this point, we can create the next
cpuhold thread to occupy another CPU if needed.
During this song and dance, it is critical that the 'copy' not pend. If
it pends, we can not guarantee which CPU it will execute on when it
unpends. As the cpuhold threads have the highest priority, nothing is
going to cause them to execute on another CPU for as long as they do
not pend.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
arch_interface.h is for architecture and should not be
under sys/. So move it under include/zephyr/arch/.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Add declaration of the _POSIX_C_SOURCE < 200809L for ztest library for
declaration of strtok_r and strdup functions.
Signed-off-by: Dawid Niedzwiecki <dawidn@google.com>
As we are replacing native_posix with native_sim, let's
refer to native_sim instead of native_posix in the comments.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
As per current definition of the Z_TEST macro, defined in
include/zephyr/ztest_test_new.h, the passed parameter is named
'fixture', not 'this'.
Signed-off-by: Reto Schneider <reto.schneider@husqvarnagroup.com>
This commit increases the default value of `ZTEST_TEST_DELAY_MS` from 3000
to 5000 milliseconds because the current value of 3000ms may not be
sufficient for the hosts with lower CPU clock frequency (e.g. new Zephyr CI
runners with cost-effective processors).
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This is obviously a copy & paste error. It was introduced by
107cb86bb3 (ztest: Add initial zexpect API
for delayed failing).
Signed-off-by: Reto Schneider <reto.schneider@husqvarnagroup.com>
This file uses strtok_r which is an extension to the the
std C library. Let's explicity select one of the extensions
which includes it instead of relaying on somebody having
set it for this file somewhere else.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Export the minimal set of ztest_test_* symbols to llexts so that the
zassert_* macros can be used by the extension code to affect test
execution. Calling these also requires vprintk() to be exported.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
- Support for listing both testcases and testsuites
- Support for running single suites or single test cases
- Support shuffling tests and repeating execution based on command line
arguments.
For example, build with
west build -p -b qemu_cortex_m3 tests/kernel/sleep -t run -- \
-DCONFIG_ZTEST_SHUFFLE=y -DCONFIG_ZTEST_SHELL=y
Following commands are available:
uart:~$ ztest
ztest - Ztest commands
Subcommands:
run-all :Run all tests
shuffle :Shuffle tests
list-testsuites :List all test suites
list-testcases :List all test cases
run-testsuite :Run test suite
run-testcase :Run testcase
shuffle accepts two arguments --suite_iter and --case_iter which allows
repeated exercution of testcases or suites.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Many functions return non-zero return codes on errors. I added an assert
for the case, when a function is expected to fail. It is just a
shorthand for `zassert_not_equal(0, ret);`, analogous to
`zassert_ok`, introduced in c5d85e175f.
I also added the corresponding `zassume_nok` and `zexpect_nok`.
Signed-off-by: Greter Raffael <rgreter@baumer.com>
Compiler can't tell that k_thread_abort() won't return and issues a
warning unless we tell it that control never gets this far.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
rand32.h does not make much sense, since the random subsystem
provides more APIs than just getting a random 32 bits value.
Rename it to random.h and get consistently with other
subsystems.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Rather than yielding to idle thread, keep the part awake.
So debugger can still access it,
since some SOCs cannot be debugged in low power states.
Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
Flaky tests give innacurate test summary indicating a misleading
passing test suite. Add new status for flaky tests.
Signed-off-by: Al Semjonovs <asemjonovs@google.com>
