The filter here should be used to filter for capability:
whether the platform configuration supports userspace. And if
it does support userspace, we then enable CONFIG_TEST_USERSPACE
(and thus CONFIG_USERSPACE) for testing. We should not be
filtering for whether userspace is enabled, but should really
be filtering for whether userspace is supported.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
For tests that support both targeting unit_testing and
other targets, we check in the cmake code the BOARD variable.
Let's allow users to set this to either of unit_testing
or unit_testing/unit_testing so it behaves like for other
tests.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
native_posix is now deprecated.
Building these tests in both native_sim and native_posix does not
improve coverage for the tests or subsystem but doubles CI time.
As anyhow native_posix will be removed all together in
2 releases, let's remove it already for these.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Many functions return non-zero return codes on errors. I added an assert
for the case, when a function is expected to fail. It is just a
shorthand for `zassert_not_equal(0, ret);`, analogous to
`zassert_ok`, introduced in c5d85e175f.
I also added the corresponding `zassume_nok` and `zexpect_nok`.
Signed-off-by: Greter Raffael <rgreter@baumer.com>
- Add integration_platforms to avoid excessive filtering
- Make sure integration platforms are actually part of the filter
- Fix some tags and test meta data
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Twister now supports using YAML lists for all fields that were written
as space-separated lists. Used twister_to_list.py script. Some artifacts
on string length are due to how ruamel dumps content.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
This commit updates all deprecated `CONFIG_LIB_CPLUSPLUS` usages to:
* check if the Zephyr minimal C++ library is enabled using
`CONFIG_MINIMAL_LIBCPP` instead of relying on the
`CONFIG_LIB_CPLUSPLUS`-based inference.
* select `CONFIG_REQUIRES_FULL_LIBCPP` when there exists a component-
level C++ standard library dependency. This allows a component to
declare C++ standard library dependency without designating a
specific libray implementation.
* select the correct type of C++ standard library implementation to use
through one of the `CONFIG_LIBCPP_IMPLEMENTATION` choices.
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
This commit updates all in-tree code to use `CONFIG_CPP` instead of
`CONFIG_CPLUSPLUS`, which is now deprecated.
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
The old random timer test was not random-looking
enough on some platforms.
Replace with new test which is psuedo-xoshiro.
The generator is still deterministic
and does not depend on entropy at all,
but should look more random for testing.
Change name of generator tree-wide also.
Signed-off-by: Declan Snyder <declan.snyder@nxp.com>
When CONFIG_ZTEST_SHUFFLE is enabled and ASSERTS are disabled
`start_pos` becomes an unused variable leading to a compile error.
Cleaned-up shuffling algorithm to not need a `start_pos` check.
Signed-off-by: Al Semjonovs <asemjonovs@google.com>
With the support of Kconfig in unittest CMake module then prj.conf is
now supported.
Add a prj.conf with ZTEST config settings and remove hardcoded values
from CMakeLists.txt file.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Updates the ztest_test_fail() function to allow failures in setup.
When executed, a failed assert will fail every test in the suite owning
the setup function. This was verified by adding a suite which asserts
in the setup function and has a test that should pass. During
exeuction, ztest marks the test as failing.
In order to verify exection I also added 2 new APIs:
- ZTEST_EXPECT_FAIL(suite_name, test_name)
- ZTEST_EXPECT_SKIP(suite_name, test_name)
Signed-off-by: Yuval Peress <peress@google.com>
Add a bunch of missing "zephyr/" prefixes to #include statements in
various test and test framework files.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Fix the deprecated calls to loading ZephyrUnittest by replacing them
with `Zephyr COMPONENTS unittest`.
Signed-off-by: Yuval Peress <peress@google.com>
Adds few missing zephyr/ prefixes to leftover #include statements that
either got added recently or were using double quote format.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Enable ZTEST_DO_THE_SHUFFLE to shuffle the order tests are ran.
Additional configs ZTEST_DO_THE_SHUFFLE_SUITE_REPEAT_COUNT
ZTEST_DO_THE_SHUFFLE_TEST_REPEAT_COUNT specify the number of times
the test or suite is executed.
Signed-off-by: Al Semjonovs <asemjonovs@google.com>
Some names of the test cases are duplicated within the project.
This commit contains the proposed names of the test scenarios.
Signed-off-by: Katarzyna Giadla <katarzyna.giadla@nordicsemi.no>
1. Test suites in prior ztest serve no purpose other than logical
ordering of tests into a named-group. Move the construct of setup and
teardown into the test suite and away from individual tests.
Additionally, add the constructs of before/after to the test suites.
This model more closely resembels other testing frameworks such as gTest
and Junit.
2. Test can be added to a suite by using ZTEST() or ZTEST_F() where _F
stands for fixture. In the case where _F is used, the argument `this`
will be provided with the type `struct suite_name##_fixture*`. Again,
this models other modern testing frameworks and allows the test to
directly access the already set up data related to the test suite.
3. Add the concept of test rules (from Junit). Rules are similar to the
before/after functions of the test suites but are global and run on all
suites. An example of a test rule can be to check that nothing was
logged to ERROR. The rule can cause the test to fail if anything was
logged to ERROR during an integration test. Another example would be a
rule that verifies that tests ran within some defined timeout.
Signed-off-by: Yuval Peress <peress@google.com>
Move to CMake 3.20.0.
At the Toolchain WG it was decided to move to CMake 3.20.0.
The main reason for increasing CMake version is better toolchain
support.
Better toolchain support is added in the following CMake versions:
- armclang, CMake 3.15
- Intel oneAPI, CMake 3.20
- IAR, CMake 3.15 and 3.20
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This commit is a followup to PR #25808 which updates the tests to
ensure the REQUIRED keyword is also used for the ZephyrUnitest package.
This provides a better error message when building with CMake and
forgetting ZEPHYR_BASE or not registering Zephyr in the CMake package
registry.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
... because it is (required).
This makes a difference when building with CMake and forgetting
ZEPHYR_BASE or not registering Zephyr in the CMake package registry.
In this particular case, REQUIRED turns this harmless looking log
statement:
-- Could NOT find Zephyr (missing: Zephyr_DIR)
-- The C compiler identification is GNU 9.3.0
-- The CXX compiler identification is GNU 9.3.0
-- Check for working C compiler: /usr/bin/cc
-- ...
-- ...
-- ...
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:8 (target_sources):
Cannot specify sources for target "app" which is not built by
this project.
... into this louder, clearer, faster and (last but not least) final
error:
CMake Error at CMakeLists.txt:5 (find_package):
Could not find a package configuration file provided by "Zephyr" with
any of the following names:
ZephyrConfig.cmake
zephyr-config.cmake
Add the installation prefix of "Zephyr" to CMAKE_PREFIX_PATH or set
"Zephyr_DIR" to a directory containing one of the above files. If
"Zephyr" provides a separate development package or SDK, be sure it
has been installed.
-- Configuring incomplete, errors occurred!
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
The following commit:
commit 407b49b35c (refs/bisect/bad)
Author: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Date: Wed Feb 12 15:00:46 2020 +0100
cmake: use find_package to locate Zephyr
breaks as we don't find the ZephyrUnittest package. For now revert to
the old means until a proper fix can be made.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Using find_package to locate Zephyr.
Old behavior was to use $ENV{ZEPHYR_BASE} for inclusion of boiler plate
code.
Whenever an automatic run of CMake happend by the build system / IDE
then it was required that ZEPHYR_BASE was defined.
Using ZEPHYR_BASE only to locate the Zephyr package allows CMake to
cache the base variable and thus allowing subsequent invocation even
if ZEPHYR_BASE is not set in the environment.
It also removes the risk of strange build results if a user switchs
between different Zephyr based project folders and forgetting to reset
ZEPHYR_BASE before running ninja / make.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This commit changes zassert_mem_equal as a macro instead
of a function implementation.
In the previous implementation when an assertion fails
the location inside ztest_assert.h file was displayed.
This modification displays the location where zassert_mem_equal
was used.
Signed-off-by: Radoslaw Koppel <radoslaw.koppel@nordicsemi.no>
Update the files which contain no license information with the
'Apache-2.0' SPDX license identifier. Many source files in the tree are
missing licensing information, which makes it harder for compliance
tools to determine the correct license.
By default all files without license information are under the default
license of Zephyr, which is Apache version 2.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Move test related code and the testsuite away from tests/ and make it a
proper subsystem.
The way tests were integrate in the tree was not obvious and actual
tests were intermixed with the testsuite code.
This will allow us to have trees with the testcode and without the
samples by just remove the folders tests/ and samples, needed for
isolating actual code from test/sample code.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
