Update this page to account for the fact that now
we have several bsim boards in tree.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
The native device type name is "native", "native posix" is just
one of the possible target boards (which is going to be deprecated
in the next release).
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Added pytest_dut_scope keyword under harness_config section.
New keyword is used to determine the scope of dut and shell
fixtures in pytest-twister-harness plugin.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
- Added language info to code-blocks when relevant (ex. for yaml
snippets)
- Formatted grammar for expression language as antlr and added
syntax highlighting
- Fixed various formatting issues with lists, definition lists...
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Document how to use pytest_root keyword. Update docs
with changes in pytest-twister-harness plugin.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
the Test Framework example present in documentation does not work
out of the box and has to be slightly modified. This is an attempt
to fix the example so it would just work directly from documentation.
Signed-off-by: Andrei Hutanu <andrei.hutanu.i@gmail.com>
We use ZTEST_DMEM and ZTEST_BMEM for running tests with memory shared
between userspace and kernelspace. This ought to be documented.
Document ZTEST_DMEM and ZTEST_BMEM.
Signed-off-by: Aaron Massey <aaronmassey@google.com>
Enable the possibility for boards to implement a custom `run` target in
its board.cmake to run any arbitrary commands. This is helpful for devs
who would like to add support for proprietary simulator to their boards
that can't be upstreamed.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Add DeviceAbstract class to default imports from pytest-twister-harness
package to simplify importing DUT package, when creating tests.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Add support for calling the `renode-test` command from west and twister.
Enable running Robot Framework tests suites in Renode.
Signed-off-by: Michał Szprejda <mszprejda@antmicro.com>
Signed-off-by: Mateusz Hołenko <mholenko@antmicro.com>
Some 3P logic (Pigweed for example) is already heavily invested in
gTest as a testing framework. Adding the `gtest` harness enables
running the existing 3P tests on various Zephyr platforms and configs.
Signed-off-by: Yuval Peress <peress@google.com>
For users convenience, as it is a typical usecase to
want to see the actual coverage of a set of tests.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
About how do we use it in general, and with lots of links so
people can find more details.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Use explicit flash command execution timeout at DeviceHandler
instead of the hardcoded value of 60 sec.
When a HW platform executes test cases right after the flash command,
the test case remaining timeout is affected by how much time the flash
operation consumes. Some simulation platforms need to spend significant
amount of time on each 'flash' cycle, usually adding the same delay on
each test case.
This improvement adds two new command line options and device map fields:
--device-flash-timeout - for the flash operation timeout
('flash-timeout' device map field).
--device-flash-with-test - to indicate that the platform flash
command also runs a test case, so the overall timeout should be
calculated as a sum of the flash timeout and the current test case
timeout to receive all console output from the platform
('flash-with-test' device map field).
The device map field values override command line values for the
particular platform where configured.
Default behavior is backward compatible: flash operation fixed timeout
is 60 sec. not including the test case timeout.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
This fixes a typo in the testing framework docs where the
ZTEST_EXPECT_SKIP MACRO seems to be referring to the wrong function.
Signed-off-by: Ivan Herrera Olivares <ivan.herreraolivares@gmail.com>
Add supplementary header <zephyr/fff_extensions.h>. Add macro
to fff_extensions.h for simplifying definition of custom fake functions
needing call-unique information for producing desired output data.
When an array of custom fake context structures is defined and
the return field within the first structure instance is registered
with the standard SET_RETURN_SEQ() macro of FFF, the
RETURN_HANDLED_CONTEXT() macro provides the inverse logic to
recover the context structure for this called instance. The body of
the custom fake handler is provided to the RETURN_HANDLED_CONTEXT()
macro for appropriate execution and access to the custom fake
parameters.
A test suite is also provided to verify macro implementation and
illustrate usage. It is at:
zephyr/tests/subsys/testsuite/fff_fake_contexts/
This code was verified by:
1. (Pass) west build -p always \
-b unit_testing tests/subsys/testsuite/fff_fake_contexts/ && \
./build/testbinary
2. (Pass) west build -p always \
-b native_posix tests/subsys/testsuite/fff_fake_contexts/ && \
./build/zephyr/zephyr.exe
3. (Pass) ./scripts/twister -p unit_testing \
-T tests/subsys/testsuite/fff_fake_contexts/
4. (Pass) ./scripts/twister -p native_posix \
-T tests/subsys/testsuite/fff_fake_contexts/
5. (Pass) cd doc && build html-fast
Fix#55246
Signed-off-by: Gregory Shue <gregory.shue@legrand.com>
This page is:
- using literal blocks [1] where it should be using code
blocks [2], which won't produce the desired results [3]
- missing required whitespace between code block directives and their
pygments syntax arguments [4]
In other words, this:
.. code-block::foo
should be:
.. code-block:: foo
- using an incorrect syntax (option:`CONFIG_FOO` or
:kconfig:option:`CONFIG_FOO=y1) to refer to Kconfig
options (should be :kconfig:option:`CONFIG_FOO`, etc)
Fix these issues so we get syntax-highlighted code blocks in the
correct pygments highlight syntax and valid references to Kconfig
options.
Adjust directive content indentation to match the usual .rst style and
make some grammar tweaks as well.
[1] https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#literal-blocks
[2] https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#directive-code-block
[3] Use of literal blocks can produce the same results if there is a
consistent choice of highlight language specified for the file, but
this file is intermixing code blocks that should be highlighted as
either C or shell commands, so there is no single choice of
highlight syntax that works.
[4] https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html#directives
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Twister allows filtering based on kconfigs and dts, however the
filtration is a part of the cmake stage, i.e. the stage has to pass
first and then twister checks if required properties are available.
This causes problems, when the full cmake stage is unable to pass.
If so, other filtration methods had to be used, e.g. platform_allow.
The commit modifies the twister workflow:
if a test defines filters based on kconfig/dts first a cmake package
helper script is used to extract dt and/or kconfigs and if the
conditions are fulfilled it proceeds to a regular cmake stage.
If not, test is skipped.
Co-authored-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
Add the zexpect API, inspired by GoogleTest's EXPECT API. This API reports
test failures while allowing test execution to continue. This enables test
reports to show more than a singule failing property on a failing test.
Signed-off-by: Aaron Massey <aaronmassey@google.com>
This commit documents the new SCA tool infrastructure.
The existing documentation for sparse are relocated into a dedicated
documentation folder for SCA tools.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Adds an option to inform twister a testsuite should only be built and
run for platforms with unique sets of attributes. This enables
for example keying on unique (arch, simulation) platforms to run the test
suite on.
The most common usage may be test suites configured to run once per
(arch, simulation) pair as being enough. Additional information about
platforms may enable running a test once per hardware IP block or once
per soc family or soc avoiding duplicated effort in building and running
tests when once suffices.
Signed-off-by: Tom Burdick <thomas.burdick@intel.com>
We want to be able to have platform or architecture extra configs
without having to duplicate a whole section of the test specification.
This adds support for namespacing of extra configs, for example:
arch:nios2:CONFIG_SYS_CLOCK_TICKS_PER_SEC=1000
or
platform:qemu_x86:CONFIG_FOO=y
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When using llvm we need to set the gcov-tool to "llvm-cov gcov" but
the lcov tool is incapable of passing arguments to the gcov-tool. i.e.
the following cannot work:
$ lcov --gcov-tool "llvm-cov gcov" ...
Instead, create a symlink to llvm-cov prefixed as `gcov` which by the
documentation of llvm-cov will alias to `llvm-cov gcov` subcommand.
Signed-off-by: Yuval Peress <peress@google.com>
Add sysbuild flag to twister supported options, with documentation on
how Kconfig and devicetree will be parsed
Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
Update the Sparse documentation to refer to the official Sparse
documentation instead of some random link, which no longeer works.
It also fixes an incorrect inline literal usage.
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
