Some names of the test cases are duplicated within the project.
This commit contains the proposed names of the test scenarios.
Signed-off-by: Katarzyna Giadla <katarzyna.giadla@nordicsemi.no>
The filter in the testcase.yaml made it impossible for these to run.
Also, the Kconfig names for the options didn't really make much sense
so they were renamed to avoid conflict.
This fixes an issue found by #36433.
Signed-off-by: Yuval Peress <peress@google.com>
Test were executed on single CPU only and with qemu_cortex_a9
excluded. Removing those limitations after fixes.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
Seen failures on some platforms. No harm to relax the
check for test timeout.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
Change the CPU_CORTEX_R kconfig option to CPU_AARCH32_CORTEX_R to
distinguish the armv7 version from the armv8 version of Cortex-R.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
1. Test suites in prior ztest serve no purpose other than logical
ordering of tests into a named-group. Move the construct of setup and
teardown into the test suite and away from individual tests.
Additionally, add the constructs of before/after to the test suites.
This model more closely resembels other testing frameworks such as gTest
and Junit.
2. Test can be added to a suite by using ZTEST() or ZTEST_F() where _F
stands for fixture. In the case where _F is used, the argument `this`
will be provided with the type `struct suite_name##_fixture*`. Again,
this models other modern testing frameworks and allows the test to
directly access the already set up data related to the test suite.
3. Add the concept of test rules (from Junit). Rules are similar to the
before/after functions of the test suites but are global and run on all
suites. An example of a test rule can be to check that nothing was
logged to ERROR. The rule can cause the test to fail if anything was
logged to ERROR during an integration test. Another example would be a
rule that verifies that tests ran within some defined timeout.
Signed-off-by: Yuval Peress <peress@google.com>
Because RISC-V arch (M extension) doesn't trigger exception for
division-by-zero, this test can't support RISC-V and is disabled in
RISC-V.
Signed-off-by: Jim Shu <cwshu@andestech.com>
Introduce a weak implementation of test_main() which calls:
* ztest_run_registered_test_suites(NULL);
* ztest_verify_all_registered_test_suites_ran();
This will attempt to run all registered test suites and verify that
they each ran.
Signed-off-by: Yuval Peress <peress@chromium.org>
Add new functionality to ztest to improve test modularity. The two
primary new entry points are:
* ztest_register_test_suite
* ztest_run_registered_test_suites
When registering a new test suite, users provide the name as well as
an optional predicate used to filter the tests for each run. Using NULL
as the predicate ensures that the test is run exactly once (after which
it is automatically filtered from future runs).
Calls to ztest_run_registered_test_suites take a state pointer as an
argument. This allows the the pragma functions to decide whether the
test should be run.
The biggest benefit of this system (other than the ability to filter
tests and maintain a larger test state) is the ability to better
modularize the test source code. Instead of all the various tests
having to coordinate and the main function having to know which tests
to run, each source file manages registering its own test
suite and handling the conditions for running the suite.
Signed-off-by: Yuval Peress <peress@chromium.org>
Issue #38643
Introduce a more powerful and well established mocking framework
into Zephyr. It also allows running the actual FFF tests using the
zephyr SDK and ztest framework to ensure compatibility.
As per TSC meeting, the fff.h header was directly added to
subsys/testsuite/include/. As per the guidelines, the file is exactly
the same as it is in FFF's library, but re-styled with clang-format.
The west.yml entry was added using the "ci" group and filtered by
default. (note that the tests will break until the CI actually
specifies that the group is needed).
Signed-off-by: Yuval Peress <peress@google.com>
Fix the trigger_fault_access. The fix is similar to cortex m.
Fix the trigger_fault_divide_zero. The fix is similar to qemu, i.e.
skip this testcase, because FVP would not trigger the exception for
dividing zero either.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Move to CMake 3.20.0.
At the Toolchain WG it was decided to move to CMake 3.20.0.
The main reason for increasing CMake version is better toolchain
support.
Better toolchain support is added in the following CMake versions:
- armclang, CMake 3.15
- Intel oneAPI, CMake 3.20
- IAR, CMake 3.15 and 3.20
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
The Cortex-R platforms may no use XIP which means userspace code with
have read and execute access to some of the RAM so that it can execute
code. However, userspace will not have access the the protected
read/write parts of RAM so it is safe to use the Cortex-M version of
trigger_fault_access for this test.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
On the Cortex M0+, M0, M23 the div-by-zero is not mapped
as an hardware exception but triggers a HardFault.
The test case is skipped for any ARMV6_M_ARMV8_M_BASELINE.
Signed-off-by: Francois Ramu <francois.ramu@st.com>
This compile test should be checking if a symbol has been defined,
otherwise it is using the kconfig value directly. This fixes a warning
../src/main.c:115:37: warning: "CONFIG_SOC_QEMU_ARC" is not defined,
evaluates to 0 [-Wundef]
when using the -Wundef flag.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
qemu arc has limited support for DIV/REM, no DIVZERO exception
throw, so we need to skip trigger_fault_divide_zero test case.
Signed-off-by: Watson Zeng <zhiwei@synopsys.com>
Put the testcase test_catch_assert_in_isr() to execute last, to prevent
it affects other test cases. Because when we caught an assert failure
in the ISR handler, it cannot be guaranteed that all the current
program status would be recovered.
Fixes#34844.
Signed-off-by: Enjia Mai <enjiax.mai@intel.com>
Make the divide instruction not being optimized of the ztest example
test case. Do this because it will be replaced by invalid opcode
exception instruction generated by GCC.
Fixes#33403.
Signed-off-by: Enjia Mai <enjiax.mai@intel.com>
em_starterkit has ICCM at 0x0 address, access to 0x0
address doesn't generate any exception, so we access
to 0xFFFFFFFF address instead to trigger exception.
Signed-off-by: Watson Zeng <zhiwei@synopsys.com>
When running the test on Cortex-M platforms, use access to
_current to trigger a memory access fault, as this address
is guaranteed to be in kernel ram and will not trigger
SecureFault exceptions.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
This functions is being called across the tree, no reason why it should
not be a public API.
The current usage violates a few MISRA rules.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The quirky behavior when accessing of 0x0 address is related to
exact ARC SoCs/boards and not to entire ARC EM CPUs family. So
check for exact SoCs configs (CONFIG_SOC_ARC_IOT and CONFIG_SOC_NSIM)
instead of checking against CONFIG_CPU_ARCEM.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Minor fixes to the sample output in the README file,
reflecting the correct error reason (3: oops) instead
of reason 0 (generic CPU error).
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
test_catch_z_oops test case should not run in user mode,
since Z_OOPS needs to be called in supervisor mode, otherwise
we will be getting a reguser user memory access error that is
irrelevant to the test case (and simply is triggered by Z_OOPS
accessing the thread's syscall frame pointer).
In addition to that, we fix the argument of Z_OOPS, because, it
was triggering a null-pointer dereferencing, which results in an
error thrown before Z_OOPS is even executed (if null-pointer
exception is caught).
We also need to generate a dummy thread->syscall_frame, that
Z_OOPS implementation will access. We fix this to
_image_ram_start, because this memory is always part of the
image memory and is always available to supervisor mode. So,
accessing it won't trigger a security fault, for example.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
The test case suggests that "For most arch which support
userspace, derefencing NULL pointer will be caught by
exception.". This is certainly not true for Cortex-M, where
read access to address 0x0 is generally allowed. The reason
the test had been passing was either 1) because in many
Cortex-M platforms, including QEMU, address 0x0 is unmapped,
or 2) GCC is generating an undefining instruction. However,
now that we have activated the null pointer derefrencing
detection, we may end up with two exceptions and the test
would fail.
Change illegal access to something outside the mapped memory
area, e.g. 0xFFFFFFFF.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Enlarge the test thread stack size, to fix test case fail
in some board which need more stack size for testing, when
CONFIG_EXCEPTION_DEBUG is enabled.
Signed-off-by: Enjia Mai <enjiax.mai@intel.com>
Fix issue #31339 and #31419, test case fail due to access NULL did not
trigger a fatal error in some platform such as nsim_em and iotdk.
Signed-off-by: Enjia Mai <enjiax.mai@intel.com>
This commit is a followup to PR #25808 which updates the tests to
ensure the REQUIRED keyword is also used for the ZephyrUnitest package.
This provides a better error message when building with CMake and
forgetting ZEPHYR_BASE or not registering Zephyr in the CMake package
registry.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
... because it is (required).
This makes a difference when building with CMake and forgetting
ZEPHYR_BASE or not registering Zephyr in the CMake package registry.
In this particular case, REQUIRED turns this harmless looking log
statement:
-- Could NOT find Zephyr (missing: Zephyr_DIR)
-- The C compiler identification is GNU 9.3.0
-- The CXX compiler identification is GNU 9.3.0
-- Check for working C compiler: /usr/bin/cc
-- ...
-- ...
-- ...
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:8 (target_sources):
Cannot specify sources for target "app" which is not built by
this project.
... into this louder, clearer, faster and (last but not least) final
error:
CMake Error at CMakeLists.txt:5 (find_package):
Could not find a package configuration file provided by "Zephyr" with
any of the following names:
ZephyrConfig.cmake
zephyr-config.cmake
Add the installation prefix of "Zephyr" to CMAKE_PREFIX_PATH or set
"Zephyr_DIR" to a directory containing one of the above files. If
"Zephyr" provides a separate development package or SDK, be sure it
has been installed.
-- Configuring incomplete, errors occurred!
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
The following commit:
commit 407b49b35c (refs/bisect/bad)
Author: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Date: Wed Feb 12 15:00:46 2020 +0100
cmake: use find_package to locate Zephyr
breaks as we don't find the ZephyrUnittest package. For now revert to
the old means until a proper fix can be made.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Using find_package to locate Zephyr.
Old behavior was to use $ENV{ZEPHYR_BASE} for inclusion of boiler plate
code.
Whenever an automatic run of CMake happend by the build system / IDE
then it was required that ZEPHYR_BASE was defined.
Using ZEPHYR_BASE only to locate the Zephyr package allows CMake to
cache the base variable and thus allowing subsequent invocation even
if ZEPHYR_BASE is not set in the environment.
It also removes the risk of strange build results if a user switchs
between different Zephyr based project folders and forgetting to reset
ZEPHYR_BASE before running ninja / make.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
A Kconfig boolean is added to allow users to provide their own
output strings when running tests via ztest.
This allows changing e.g. the PASS/FAIL/SKIPPED strings,
add counters, change separators, and similar.
A test using the feature and relevant documentation is added.
Signed-off-by: Torstein Grindvik <torstein.grindvik@nordicsemi.no>
This commit changes zassert_mem_equal as a macro instead
of a function implementation.
In the previous implementation when an assertion fails
the location inside ztest_assert.h file was displayed.
This modification displays the location where zassert_mem_equal
was used.
Signed-off-by: Radoslaw Koppel <radoslaw.koppel@nordicsemi.no>
