The `handle_signal()` function in the new ztest API (`ztest_new.c`) uses
an array of strings to lookup a friendly name for each test phase, but
the array only has three elements despite there being six test phases.
This can lead to an out-of-bounds issue. Replace the array with a helper
function and switch statement.
Signed-off-by: Tristan Honscheid <honscheid@google.com>
This change introduced some weird failures on x86 that will take some
time to untangle, and wasn't a particularly important feature to
merge. Revert for now.
This reverts commit adc901aa6a.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Some hardware has "interesting" configuration like asymmetric default
interrupt masking (the intel_adsp devices in particular, but x86's
IO-APIC driver has tripped over this in the past too) that needs
special treatment if you want to run something on "core 1"
specifically, and 1cpu test cases pretty much by definition are going
to have been written without SMP details in mind. Switch the logic
around a tiny bit such that these test cases always run on CPU ID zero
explicitly.
Actually in practice this was ALMOST guaranteed to be true already,
because test setup and happens serially, having been started on the
main thread, which starts on CPU 0 by definition. Then the test
teardown aborts all the spawned threads that might have been running
on CPUs 1+, so those reach idle, and the next test case starts
syncronously on the same thread (and thus CPU) where it started. But
nonetheless that wasn't actually enforced, and we've found at least
one simulation environment where timing conspires to break things.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Some SOCs cannot be flashed reliably in low power modes. If
CONFIG_ZTEST_NO_YIELD is selected, do not yield to the idle thread after
testsuite completes, so that the SOC will not enter low power mode.
Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
Add ztest_test_skip() support to non-kernel tests by implementing
long jump buffer and TC_SKIP result collection.
Signed-off-by: Thomas Ebert Hansen <thoh@oticon.com>
When system clock is set globally for the test which is executed
on various qemu platforms it may happen that clock is set too
high compared to CPU clock frequency. In that case test may stuck.
Added check and test skipping in that case.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
Fix an issue where a test's suite name was misspelled and thus the
test never ran because it wasn't associated with a suite. Example:
ZTEST_SUITE(my_suite, NULL, NULL, NULL, NULL, NULL);
ZTEST(myy_suite, test)
{
}
The above will no longer pass since `myy_suite` with a double `y` will
fail the verification.
Signed-off-by: Yuval Peress <peress@google.com>
Use UTIL_CAT which resolves input arguments instead of direct
preprocessor concatenation.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
Flushing logs here is especially vital if LOG_PRINTK is enabled.
In that case printk goes to log and without flushing after all
tests are run you will not get the final report out.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
A lot of tests need to be able to get their current tid and do some
action with it. It makes sense for the `before` function/rule to be
able to run in the same thread as the test. Note that the `after`
function does not run in the same thread because we need to guarantee
that it will run.
Signed-off-by: Yuval Peress <peress@google.com>
Using macro which will resolve complex SUITE_NAME. Without
it wrong name was assigned when SUITE_NAME was consists of
concatenated defines.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
Add maximum timeout used for conversion to Kconfig. Option is used
to determine which conversion algorithm to use: faster but overflowing
earlier or slower without early overflow.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
Fix initial thread priority and move starting of the
test timer after threads initialization to ensure that
it does not expire before threads are initiated.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
Currently, ztress is not supporting SMP, mainly because of
cpu load calculation algorithm. It may be re-enabled in the
future.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
1. Test suites in prior ztest serve no purpose other than logical
ordering of tests into a named-group. Move the construct of setup and
teardown into the test suite and away from individual tests.
Additionally, add the constructs of before/after to the test suites.
This model more closely resembels other testing frameworks such as gTest
and Junit.
2. Test can be added to a suite by using ZTEST() or ZTEST_F() where _F
stands for fixture. In the case where _F is used, the argument `this`
will be provided with the type `struct suite_name##_fixture*`. Again,
this models other modern testing frameworks and allows the test to
directly access the already set up data related to the test suite.
3. Add the concept of test rules (from Junit). Rules are similar to the
before/after functions of the test suites but are global and run on all
suites. An example of a test rule can be to check that nothing was
logged to ERROR. The rule can cause the test to fail if anything was
logged to ERROR during an integration test. Another example would be a
rule that verifies that tests ran within some defined timeout.
Signed-off-by: Yuval Peress <peress@google.com>
Added framework for concurrency testing. Framework setup multiple
priority contexts and executes user handlers in that context. Test
terminates after certain number of repetitions or preemptions or
when timeout occurs. It can also be aborted by the user.
Framework can be used for testing resiliency to preemptions.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
Add guard to not pend until all logs are flushed when default
logging thread is disabled. In that case, logging has no control
where logs are processed and sleeping may not lead to flushing
log data. That may result in test hanging.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
This changes both k_mem_domain_add_partition() and
k_mem_domain_remove_partition() to return errors instead of
asserting when errors are encountered. This gives the application
chance to recover.
The arch_mem_domain_parition_add()/_remove() will be modified
later together with all the other arch_mem_domain_*() changes
since the architecture code for partition addition and removal
functions usually cannot be separately changed.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When deferred mode is used, logging is using lowest priority
thread to process the logs. When test cases are performed one
by one processor never reaches lowest priority thread until
whole suite is completed. Added flushing after each test case.
Feature is optional (by default enabled).
CONFIG_TEST_LOGGING_FLUSH_AFTER_TEST=n to disable.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
This change adds `k_cycle_get_64()` on platforms that
support a 64-bit cycle counter.
The interface functions `arch_k_cycle_get_64()` and
`sys_clock_cycle_get_64()` are also introduced.
Fixes#39934
Signed-off-by: Christopher Friedt <chrisfriedt@gmail.com>
Introduce a weak implementation of test_main() which calls:
* ztest_run_registered_test_suites(NULL);
* ztest_verify_all_registered_test_suites_ran();
This will attempt to run all registered test suites and verify that
they each ran.
Signed-off-by: Yuval Peress <peress@chromium.org>
Add new functionality to ztest to improve test modularity. The two
primary new entry points are:
* ztest_register_test_suite
* ztest_run_registered_test_suites
When registering a new test suite, users provide the name as well as
an optional predicate used to filter the tests for each run. Using NULL
as the predicate ensures that the test is run exactly once (after which
it is automatically filtered from future runs).
Calls to ztest_run_registered_test_suites take a state pointer as an
argument. This allows the the pragma functions to decide whether the
test should be run.
The biggest benefit of this system (other than the ability to filter
tests and maintain a larger test state) is the ability to better
modularize the test source code. Instead of all the various tests
having to coordinate and the main function having to know which tests
to run, each source file manages registering its own test
suite and handling the conditions for running the suite.
Signed-off-by: Yuval Peress <peress@chromium.org>
According to the rule MISRAC-2012 21.4.a the standard header
file <setjmp.h> shall not be used. Suppress it, because it raises
violation in a testcode, not in a runtime code.
Tag suppresses reporting of violation for the current file,
starting from the line where the suppression is located.
It is a deliberate deviation.
Found as a coding guideline violation (MISRA R21.4.a) by static
coding scanning tool.
Signed-off-by: Maksim Masalski <maksim.masalski@intel.com>
According to the rule MISRAC-2012 21.4.b the standard header
file <setjmp.h> shall not be used. Suppress it, because it raises
violation in a testcode, not in a runtime code.
Tag suppresses reporting of violation for the current file,
starting from the line where the suppression is located.
Found as a coding guideline violation (MISRA R21.4.b) by static
coding scanning tool.
Signed-off-by: Maksim Masalski <maksim.masalski@intel.com>
This adds FPU sharing support with a lazy context switching algorithm.
Every thread is allowed to use FPU/SIMD registers. In fact, the compiler
may insert FPU reg accesses in anycontext to optimize even non-FP code
unless the -mgeneral-regs-only compiler flag is used, but Zephyr
currently doesn't support such a build.
It is therefore possible to do FP access in IRS as well with this patch
although IRQs are then disabled to prevent nested IRQs in such cases.
Because the thread object grows in size, some tests have to be adjusted.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Added support for no multithreading case where test cases are
called directly from main(). On failure in ztest assert macro,
macro returns from the function. It implies that ztest assert
macros can only be called in the test function.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
Reboot functionality has nothing to do with PM, so move it out to the
subsys/os folder.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
We have a few remaining tests where ztest module is not used directly,
and instead lower-level tc_util.h is used (where ztest also uses that
header). Supposedly, there're good reasons for that. However, tc_util.h
tests have output which is somewhat inconsistent with ztest output,
which may be a problem with automated parsing of test results, e.g. in
CI systems.
So, factor out code to mark testsuite start/end from ztest.c to
tc_util.h as TC_SUITE_START() and TC_SUITE_END() macros, to allow
tc_util.h based tests to produce output fully consistent with
ztest, while avoiding duplicate of code. TC_SUITE_END() accepts
result code (TC_PASS/TC_FAIL), similar to existing TC_END_REPORT().
Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
cleanup_test() was missing final else statement in
the if else if construct. This commit adds else {}
to comply with coding guideline 15.7.
Signed-off-by: Jennifer Williams <jennifer.m.williams@intel.com>
By default ztest thread is running at the priority `-1`. This value is
invalid when the testcase is running in cooperative mode only. Set
default ztest thread priority to `-2` if this is the case. The fix is
modeled on the approach used to define the default
`MAIN_THREAD_PRIORITY`.
Signed-off-by: Piotr Mienkowski <piotr.mienkowski@gmail.com>
This functions is being called across the tree, no reason why it should
not be a public API.
The current usage violates a few MISRA rules.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Without this fix, ztest_set_assert_valid() can only be used when
CONFIG_ZTEST_FATAL_HOOK is set.
Signed-off-by: Reto Schneider <reto.schneider@husqvarnagroup.com>
zassert_within should also compare on equality instead
of only greater/lower.
example:
zassert_within(1,1,0); // should return true
zassert_within(1,2,1); // should return true
Signed-off-by: Alexander Wachter <alexander@wachter.cloud>
This is in order to reduce the redundancy code writing for fatal and
assert handler for error case testing. They can be used both in kernel
and userspace, and are also SMP safe.
Signed-off-by: Enjia Mai <enjiax.mai@intel.com>
Many functions signal success by returning a return code of zero. Add
an assertion for this.
Some functions can return a positive value to indicate success, where
they want to convey some information. This is not handled by this
assert function.
Signed-off-by: Simon Glass <sjg@chromium.org>
