Fixes bad usage of single backticks in lieu of double backticks for
rendering inline literals, or simple '*' for italics.
When appropriate, a better construct than double backticks has been
selected (ex. :file:, :kconfig:option:, :c:func:, ...), or proper :ref:
have been used if the original intention was to have a link.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Expanding the ZTEST shell to include a command that
accepts additional parameters, such as ztest run_testcase
<suite:test> -p any_value.
Adding the new ZTEST_P macro, to create the
parameterized test.
Signed-off-by: Arkadiusz Cholewinski <arkadiuszx.cholewinski@intel.com>
Add a diagram to illustrate a test application's project
components with relationships between Ztest and Twister
objects.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Remove legacy use of 'test case' and replace with `test scenario` where
applicable.
Update terminology and some other parts of twister docs.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
- Support for listing both testcases and testsuites
- Support for running single suites or single test cases
- Support shuffling tests and repeating execution based on command line
arguments.
For example, build with
west build -p -b qemu_cortex_m3 tests/kernel/sleep -t run -- \
-DCONFIG_ZTEST_SHUFFLE=y -DCONFIG_ZTEST_SHELL=y
Following commands are available:
uart:~$ ztest
ztest - Ztest commands
Subcommands:
run-all :Run all tests
shuffle :Shuffle tests
list-testsuites :List all test suites
list-testcases :List all test cases
run-testsuite :Run test suite
run-testcase :Run testcase
shuffle accepts two arguments --suite_iter and --case_iter which allows
repeated exercution of testcases or suites.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
In the docs replace references to native_posix with native_sim
Switch the default test platform to native_sim from native_posix
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
the Test Framework example present in documentation does not work
out of the box and has to be slightly modified. This is an attempt
to fix the example so it would just work directly from documentation.
Signed-off-by: Andrei Hutanu <andrei.hutanu.i@gmail.com>
We use ZTEST_DMEM and ZTEST_BMEM for running tests with memory shared
between userspace and kernelspace. This ought to be documented.
Document ZTEST_DMEM and ZTEST_BMEM.
Signed-off-by: Aaron Massey <aaronmassey@google.com>
This fixes a typo in the testing framework docs where the
ZTEST_EXPECT_SKIP MACRO seems to be referring to the wrong function.
Signed-off-by: Ivan Herrera Olivares <ivan.herreraolivares@gmail.com>
Add supplementary header <zephyr/fff_extensions.h>. Add macro
to fff_extensions.h for simplifying definition of custom fake functions
needing call-unique information for producing desired output data.
When an array of custom fake context structures is defined and
the return field within the first structure instance is registered
with the standard SET_RETURN_SEQ() macro of FFF, the
RETURN_HANDLED_CONTEXT() macro provides the inverse logic to
recover the context structure for this called instance. The body of
the custom fake handler is provided to the RETURN_HANDLED_CONTEXT()
macro for appropriate execution and access to the custom fake
parameters.
A test suite is also provided to verify macro implementation and
illustrate usage. It is at:
zephyr/tests/subsys/testsuite/fff_fake_contexts/
This code was verified by:
1. (Pass) west build -p always \
-b unit_testing tests/subsys/testsuite/fff_fake_contexts/ && \
./build/testbinary
2. (Pass) west build -p always \
-b native_posix tests/subsys/testsuite/fff_fake_contexts/ && \
./build/zephyr/zephyr.exe
3. (Pass) ./scripts/twister -p unit_testing \
-T tests/subsys/testsuite/fff_fake_contexts/
4. (Pass) ./scripts/twister -p native_posix \
-T tests/subsys/testsuite/fff_fake_contexts/
5. (Pass) cd doc && build html-fast
Fix#55246
Signed-off-by: Gregory Shue <gregory.shue@legrand.com>
This page is:
- using literal blocks [1] where it should be using code
blocks [2], which won't produce the desired results [3]
- missing required whitespace between code block directives and their
pygments syntax arguments [4]
In other words, this:
.. code-block::foo
should be:
.. code-block:: foo
- using an incorrect syntax (option:`CONFIG_FOO` or
:kconfig:option:`CONFIG_FOO=y1) to refer to Kconfig
options (should be :kconfig:option:`CONFIG_FOO`, etc)
Fix these issues so we get syntax-highlighted code blocks in the
correct pygments highlight syntax and valid references to Kconfig
options.
Adjust directive content indentation to match the usual .rst style and
make some grammar tweaks as well.
[1] https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#literal-blocks
[2] https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#directive-code-block
[3] Use of literal blocks can produce the same results if there is a
consistent choice of highlight language specified for the file, but
this file is intermixing code blocks that should be highlighted as
either C or shell commands, so there is no single choice of
highlight syntax that works.
[4] https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html#directives
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Add the zexpect API, inspired by GoogleTest's EXPECT API. This API reports
test failures while allowing test execution to continue. This enables test
reports to show more than a singule failing property on a failing test.
Signed-off-by: Aaron Massey <aaronmassey@google.com>
Make code look consistent with other code blocks and remove
double block anchor. Leave code-block to have proper syntax
highlighting.
Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
Updates the ztest_test_fail() function to allow failures in setup.
When executed, a failed assert will fail every test in the suite owning
the setup function. This was verified by adding a suite which asserts
in the setup function and has a test that should pass. During
exeuction, ztest marks the test as failing.
In order to verify exection I also added 2 new APIs:
- ZTEST_EXPECT_FAIL(suite_name, test_name)
- ZTEST_EXPECT_SKIP(suite_name, test_name)
Signed-off-by: Yuval Peress <peress@google.com>
Update custom test_main example to use updated ZTEST API so
test can either be listed or ran appropriately.
Signed-off-by: Al Semjonovs <asemjonovs@google.com>
Documentation now includes:
- How to create a test suite
- How to use predicates
- How to use setup/before/after/teardown
- How to use test rules
- Direct people to use FFF instead of ztest mock
Fixes#47420
Signed-off-by: Yuval Peress <peress@google.com>
Add an assume API which works like JUnit's. Assumptions can be made
at any point before your test returns (setup, before, and during the
test). If an assumption fails, the test will be marked as skipped.
This helps avoid a cascading affect of failed tests when a base
feature breaks. The feature is expected to have tests and the tests
which depend on it should be skipped (not failed) if that feature
is broken.
Issue #42472
Signed-off-by: Yuval Peress <peress@google.com>
Enable ZTEST_DO_THE_SHUFFLE to shuffle the order tests are ran.
Additional configs ZTEST_DO_THE_SHUFFLE_SUITE_REPEAT_COUNT
ZTEST_DO_THE_SHUFFLE_TEST_REPEAT_COUNT specify the number of times
the test or suite is executed.
Signed-off-by: Al Semjonovs <asemjonovs@google.com>
