Extend Twister Harness 'recording' feature to allow multiple
regular expression patterns to extract different types of records
from test output.
Add 'merge' recording mode to collect all extracted data fields
into a single record object of the test instance.
Export to CSV file now takes all field names occurred in the collected
records, sort it alphabetically, and then use it for columns instead of
using only the first record's fields. This is done to address possible
situation when records have different set of fields.
Adjust Twister documentation and test suite to the above changes.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Add an example for the `runners` section in the zephyr/module.yml file,
and a paragraph to the west "flash and debug runners" section.
Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
Add the cmake files for running static code analysis with the Polyspace
tools in the west build. The analysis leverages the compilation database.
Options for the analysis are documented in doc/develop/sca/polyspace.rst.
Analysis results are printed as command line output and provided as CSV.
Manually tested on v4.0.0 with various sample applications.
Signed-off-by: Martin Becker <mbecker@mathworks.com>
Instead of unconditionally erasing the whole target, add
support for using the common --erase flag.
Signed-off-by: Peter Johanson <peter@peterjohanson.com>
CLion now offers a more native approach to working with Zephyr projects.
We would like to inform users about this.
Signed-off-by: Oleg Zinovyev <oleg.zinovyev@jetbrains.com>
Extend `--no-detailed-test-id` command line option: in addition to its
current behavior to exclude from a test Suite name its configuration path
prefix, also don't prefix each Ztest Case name with its Scenario name.
For example: 'kernel.common.timing' Scenario name, the same Suite name,
and 'sleep.usleep' test Case (where 'sleep' is its Ztest suite name
and 'usleep' is Ztest test name.
This way both TestSuite and TestCase names follow the same principle
having no parent object name prefix.
There is no information loss in Twister reports with this naming:
TestSuite is a container object for its TestCases, whereas TestSuite
has its configuration path as a property.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Add documentation for the get_filtered_output method in both the Python
implementation and the Integration with pytest chapter. The method is used
to filter command outputs by removing prompts and log messages, making it
easier to process shell command results.
Signed-off-by: Thomas Günther <thomas.guenther@limatica.com>
removed traling whitespace
History seems to repeat itself and we need to momentarily discourage
folks to use Python 3.13 on Windows until windows-curses releases a
PyPI package that works with it.
See https://github.com/zephyrproject-rtos/windows-curses/issues/69Fixes#81543.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
add vendor_allow and vendor_exclude to the list of filters. The vendor
data is maintained in the board.yml files.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This change introduces the ability in twister to select which
emulation/simulation tool to use on the command line.
If none is specified, it will select the first in the list.
Signed-off-by: Wilfried Chauveau <wilfried.chauveau@arm.com>
Update Test Application diagram as well as Test Scenario and
Test Case naming conventions to make them more clear and aligned
to Ztest suite name included as a part of Test Case name.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Extended hardware map to share a single board between variants.
To run tests for different variants on the same board
without re-configuring the hardware map file for each variant,
one can use a `platform` atribute as a list of names.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Currently, Twister Statuses are only useful for
the initiated, save for the very basics.
This is in opposition to the fact
that they are the main thing end users take into
account when checking their Twister run reports.
In order to make Statuses more useful for the end user,
a new documentation page has been created,
elucidating the full meaning of all Statuses.
Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
Utilize a code spell-checking tool to scan for and correct spelling errors
in various files within the `doc` directory.
Signed-off-by: Pisit Sawangvonganan <pisit@ndrsolution.com>
There is no way to know for sure the size of an image built for a test,
given that zephyr can build for so many different architectures,
platforms, software supports, etc. So it should be clear that the filter
for min_ram and min_flash on the twister testcases is an estimate, and
not a strictly well known value.
Signed-off-by: Declan Snyder <declan.snyder@nxp.com>
To use Xtensa toolchain, various environment variables must be
set so the executables can find necessary files and what core
to compile for. This becomes an annoyance when you have to
test multiple boards with different cores. You have to use
one set of environment variables per core. Twister cannot test
them all in one setting, and it is especially annoying doing
west builds. So enhance the environment variables handling so
that TOOLCHAIN_VER and XTENSA_CORE can be replaced by
TOOLCHAIN_VAR_<board> and XTENSA_CORE_<board> where <board>
is the normalized board target (think <board>.yaml). CMake
will then figure out the core ID for the toolchain to use.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The initial note about the install guide for different distributions
can be easily overlooked. Add a second note to make it more prominent
in the first step.
Signed-off-by: Timon Skerutsch <zephyr@diodes-delight.com>
Added a note that udev rules must be configured in Linux environments.
Reformatted text accordingly.
Signed-off-by: TOKITA Hiroshi <tokita.hiroshi@gmail.com>
SDK installation has been simplified by introducing the `west sdk`
command. I updated the documentation to use this.
Signed-off-by: TOKITA Hiroshi <tokita.hiroshi@gmail.com>
This updates the documentation of all the Qemu boards
to use the new `zephyr:board::` directive.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
This updates the documentation of all the Atmel boards to use
the new `zephyr:board::` directive.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
This updates the documentation of all the NXP boards to use
the new `zephyr:board::` directive.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
This updates the documentation of all the Adafruit boards to use
the new `zephyr:board::` directive.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Fixes commit 1df078158f ("doc: build: Add signing page") that removed
the `imgtool` section from the `sign.rst` page.
rimage configuration was designed in a very different way from imgtool
configuration. The rimage doc section was following the imgtool doc
section and constrating the two approaches. Now that the previous
imgtool doc section has just been removed, some words like "different"
and "instead" don't make sense anymore. Drop them.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Add `west bindesc get_offset` command to print the offset of
the descriptors inside the given image.
Signed-off-by: Yonatan Schachter <yonatan.schachter@gmail.com>
Adds a signing page under build, which contains information on
how to sign images from the build system, this removes the old
details for using west sign with imgtool from the west sign page
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
This commit uses the new .. zephyr:code-sample-category directive to
categorize code samples across the tree.
Updates existing legacy references to manually defined targets to now
use :zephyr:code-sample-category: role instead.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Document the recommended practice regarding having default definitions for
module-specific Kconfig symbols.
Signed-off-by: Johan Hedberg <johan.hedberg@silabs.com>
Adds missing code-sample directive to the Hello World sample in
preparation for upcoming changes to the Zephyr documentation that will
be leveraging the provided description and metadata.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Fixes bad usage of single backticks in lieu of double backticks for
rendering inline literals, or simple '*' for italics.
When appropriate, a better construct than double backticks has been
selected (ex. :file:, :kconfig:option:, :c:func:, ...), or proper :ref:
have been used if the original intention was to have a link.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
As there is no check (yet!) in the directive, many
.. zephyr-app-commands:: in the documentation are pointing to
non-existing sample applications, which is problematic for users who
typically expect to just copy-paste the commands and have them work.
This commit fixes all offending references to point to existing samples,
or converts them to simple ":app:" instead of ":zephyr-app:" when the
intent wasn't to refer to in-tree samples.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
The zephyr-app-commands directive can output a helpful hint to the user
when they are trying to build a sample that is in the Zephyr tree,
telling them to ensure they are in the root folder of the Zephyr repo.
Update all doc pages that were using :app: instead of :zephyr-app: so
that the hint is displayed.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Updates references to the net-tools project to refer to the correct
placement of net-tools under tools.
Signed-off-by: Florian Grandel <fgrandel@code-for-humans.de>
Add information about main(int, argc **) in places where the documentation
stated that only parameterless main could be used
Signed-off-by: Jakub Michalski <jmichalski@internships.antmicro.com>
Signed-off-by: Filip Kokosinski <fkokosinski@antmicro.com>
Added links pointing to the Dedicated image build targets
from Optimization tools page for ram_report, rom_report, footprint,
puncover, and pahole.
Reported by @markaj-nordic.
Signed-off-by: Grzegorz Ferenc <Grzegorz.Ferenc@nordicsemi.no>
Expanding the ZTEST shell to include a command that
accepts additional parameters, such as ztest run_testcase
<suite:test> -p any_value.
Adding the new ZTEST_P macro, to create the
parameterized test.
Signed-off-by: Arkadiusz Cholewinski <arkadiuszx.cholewinski@intel.com>
A common question is how twister decides on which platform to run
a given test. Added section is meant to explain those rules.
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
Extend pytest-args from configuration yaml file with args
from command line instead of overwriting.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Added python-tk to the brew install statement.
In order to use the zephyr gui tools python must also have the
"tkinter" (python tk interface) tools available.
tkinter is NOT part of 'python3' through homebrew on macOS.
fixes#76811
Signed-off-by: Eric Southam <eric.southam@lvt.com>
1.16 was not meant to be a release, and 1.14 was long ago.
Let's use as example something much more recent.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
When deprecating APIs,
let's tell users to add to one GitHub issue tracking all
deprecated APIs meant to be removed,
so the process is simpler and hopefully developers do it.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Add a new fixture, `unlaunched_dut`, which is basically the `dut`
one, but without launching the device - thus also not building the
application. It is useful for tests who need a finer control of
the building of the application, before the dut can be launched
to run it.
It will be used on a future patch, which will use it to enable LLEXT EDK
tests.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
As it was already outlined, installing Python dependencies at a system
level or user level can be dangerous. For this reason, the default setup
instructions already pointed users to use Python virtual environments. On
newer Ubuntu installations (23.04 onwards) system-level pip installs are
not allowed. Let's make Zephyr _Python-safe_ as well by removing dangerous
instructions.
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
probe-rs is a new programming and debugging tool written in Rust, supports
many probes and targets.
This commit introduces initial support for probe-rs to Zephyr.
Signed-off-by: Chen Xingyu <hi@xingrz.me>
Breathe is not very strict about the types used to reference C domain
objects (ex. referencing a struct using `:c:type:` would work, and vice
versa) but other tooling might be. This commit fixes some of such
incorrectly documented elements.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
This reverts commit a156b619d4.
Now that Sphinx 7.4 is out, it is possible to use webp images
in the PDF documentation (they're converted to png).
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Apple has released other processors, e.g. M2 or M3, so Apple Silicon is no
longer a synonym of M1.
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
Add a diagram to illustrate a test application's project
components with relationships between Ztest and Twister
objects.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Utilize a code spell-checking tool to scan for and correct spelling errors
in various files within the `doc` directory.
Signed-off-by: Pisit Sawangvonganan <pisit@ndrsolution.com>
The current version of scipts do not consider OOT boards use cases and
the tests with robot now are strict to only one robot file, which is
not realistic for real environment. This address those issues and allow
multiple testsuits at command line and lists at tests entries. It add
another test parameter to allow configure robotframework options.
Fixes: #74563
Signed-off-by: Gerson Fernando Budke <gerson.budke@ossystems.com.br>
Terms like the west workspace or even west projects and modules were not
documented prominently enough, prompting users to ask for additional
clarification. This patch attempts to resolve this by adding terms and
explanation of concepts to the glossary and west doc pages.
Fixes#67376.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
This basic guide only covers linting so far. It could be expanded in the
future to be more accurate and detailed. The current configuration
allows to navigate the code but some symbols are not recognized by the
linter, and the native compiler is always used.
Co-authored-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
Co-authored-by: Luc BEAUFILS <luc.beaufils@savoirfairelinux.com>
Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
Signed-off-by: Luc BEAUFILS <luc.beaufils@savoirfairelinux.com>
Remove legacy use of 'test case' and replace with `test scenario` where
applicable.
Update terminology and some other parts of twister docs.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
config file snippets should use "cfg" pygment to get proper highlighting
This commit updates all occurences where "kconfig" or other languages
were used.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Fix a few issues with the probes documentation for NXP boards:
- fix references to debug probe type in probes page introduction
- add an explicit "Install the host tools" step for the LPC-Link and
MCU-Link based debug probe update steps.
Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
Normally the return code of `CodeChecker analyze` and `CodeChecker parse`
is suppressed, so all the enabled commands can execute instead of
crashing the build.
Add a new option, `CODECHECKER_PARSE_EXIT_STATUS`, to permit failing the
build if `CodeChecker parse` returns non-zero.
Signed-off-by: Noah Pendleton <noah.pendleton@gmail.com>
Improve the SPDX with the current values:
- URL: extracted from `git remote`. If more than one remote, URL is not
set.
- Version: extracted from `git rev-parse` (commit id).
- PURL and CPE for Zephyr: generated from URL and version.
For zephyr, the tag is extracted, if present, and replace the commit id for
the version field.
Since official modules does not have tags, tags are not yet extracted for
modules.
To track vulnerabilities from modules dependencies, a new SBOM,
`modules-deps.spdx` was created. It contains the `external-references`
provided by the modules. It allows to easily track vulnerabilities from
these external dependencies.
Signed-off-by: Thomas Gagneret <thomas.gagneret@hexploy.com>
Extend Twister Harness recording feature to allow selected data fields,
extracted from the log by a regular expression, to be parsed into JSON
objects and eventually reported in `twister.json` as `recording` list
property of the test suite.
With this extension, log records can convey layered data structures
passed from a test image as summary results, traces, statistics, etc.
This extension also allows flexible recording structure: a test image
can output different types of data records incapsulated into a fixed
set of fields, so `recording.csv` file columns are respected, whereas
some of the columns keep strings with json-encoded semi-structured data.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Writing headers compatible with both C and C++ _and_ a range of
toolchains can be very challenging, see for instance long discussions in
commit c15f029a71 ("init.h: restore designated initializers in
SYS_INIT_NAMED()"), PR #69411 and a few others linked from there.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Update NXP flash debug probes section, to attempt to clarify the
differences between OpenSDA, LPC-Link2, and MCU-Link probes.
Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
Allow twister fixtures to contain extra information, which can be used for
test suite configuration. The extra information can be appended to existing
fixtures separated by a colon (i.e. <fixture>:<configuration>).
This is especially useful for the pytest harness, where a fixture of a
given type may need to refer to an instance of a particular piece of host
hardware needed by the pytest suite (e.g. a network interface, a UART, or a
CAN interface connected to the device under test).
Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
Introduce `robot` command for running Robot Framework test suites.
Initial implementation consists of one runner dedicated for renode-test,
which is a Renode wrapper for running Robot tests.
Signed-off-by: Michał Szprejda <mszprejda@antmicro.com>
Introduce `simulate `command for running samples on a simulator of
choice. Initial implementation consists of one runner, dedicated for
Renode.
Signed-off-by: Michał Szprejda <mszprejda@antmicro.com>
Initial integration with renode-test was introduced in commit bdf02ff,
which added support for calling the `renode-test` command from both west
and twister.
This commit removes the custom run_renode_test target used for running
Robot tests with the `west build` command and makes twister call
`renode-test` directly instead.
Signed-off-by: Michał Szprejda <mszprejda@antmicro.com>
Installation of single toolchain causes problems for customers who
are not familiar with Zephyr infrastructure. This tip is supposed
to help them.
Signed-off-by: Nikolay Agishev <agishev@synopsys.com>
Update getting started guide, release notes, and CI actions to
use Python 3.10 as minimum supported version.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
