Twister allows filtering based on kconfigs and dts, however the
filtration is a part of the cmake stage, i.e. the stage has to pass
first and then twister checks if required properties are available.
This causes problems, when the full cmake stage is unable to pass.
If so, other filtration methods had to be used, e.g. platform_allow.
The commit modifies the twister workflow:
if a test defines filters based on kconfig/dts first a cmake package
helper script is used to extract dt and/or kconfigs and if the
conditions are fulfilled it proceeds to a regular cmake stage.
If not, test is skipped.
Co-authored-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
This is an implementation of issue #48334 and adds support for
specifying additional config and device tree overlays through fields in
the testcase.yaml file, which is more readable than having to cram these
in to `extra_args`.
Consider this example which shows the original and new ways to add
config and DT overlays:
```
common:
extra_args: "CONF_FILE=a.conf;b.conf
DTC_OVERLAY_FILE=w.overlay;x.overlay OVERLAY_CONFIG=e.conf
UNRELATED=abc"
tests:
utilities.base64:
extra_conf_files:
- "c.conf"
- "d.conf"
extra_overlay_confs:
- "extra_overlay.conf"
extra_dtc_overlay_files:
- "y.overlay"
- "z.overlay"
extra_configs:
- CONFIG_SAMPLE=y
tags: base64
type: unit
```
The new fields are `extra_conf_files`, `extra_overlay_confs,
`extra_dtc_overlay_files`. Files specified in these sections are appended
to any `CONF_FILE`, `OVERLAY_CONFIG`, or `DTC_OVERLAY_FILE` fields in
`extra_args`, causing the following args being passed in to
`self.run_cmake` at `runner.py:850`:
```
['-DUNRELATED=abc',
'-DCONF_FILE=a.conf;b.conf;c.conf;d.conf',
'-DDTC_OVERLAY_FILE=w.overlay;x.overlay;y.overlay;z.overlay',
'-DOVERLAY_CONFIG=e.conf extra_overlay.conf '
'<build_dir>/twister/testsuite_extra.conf']
```
These fields can be used in the common or scenario-specific YAML
sections and will be merged in order of least to most specific:
1. config files extracted from common's extra_args
2. files listed in common's {extra_conf_files or extra_overlay_confs
or extra_dtc_overlay_files}
3. config files extracted from test scenario's extra_args
4. files listed in test scenario's {extra_conf_files or
extra_overlay_confs or extra_dtc_overlay_files}
Specifying these files in extra_args now triggers a deprecation warning,
as the direct YAML fields are preferred for readability. They will still
function for now but support will be dropped in the future. One
testcase.yaml
(`zephyr/tests/cmake/overlays/var_expansions/testcase.yaml`) is
converted to use the new fields. A follow-up PR will convert the
remaining files to the new format.
Signed-off-by: Tristan Honscheid <honscheid@google.com>
This will make it more convenient to use it from multiple different
places, which we will have a need for in the future.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
We need to have an _include_path attribute to pretty-print
this object from within pdb, for some reason. Add it.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
There's no need for _parse_node() to return the Node instance that is
its sole argument. The only user of the return value is a dead store.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This helper lets you place a node (really the entire subtree rooted at
that node) elsewhere in the devicetree. This will be useful when
adding system devicetree support, when we'll want to be able to, for
example, move the CPU cluster node selected by the current execution
domain to /cpus.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Introduce a context manager that will save some typing
when dealing with expected exceptions.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Percentage calculation was off due to a recent fix in counting and lines
were not breaking correctly on errors.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
MCUmgr no longer uses DTS node property label to identify
slots for image upload and running application. It now uses DTS node
labels for that purpose.
This commit moves `west sign` from using label properties
to DTS node labels, same way the MCUmgr does.
Signed-off-by: Dominik Ermel <dominik.ermel@nordicsemi.no>
Fixed runtime cleanup option. A typo in a conditional prevented the
cleanup from happened when a test has passed.
Fixes#54240
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
To get consistent and 1 single rundate per CI run, set run_date when
we upload the data. This is optional mostly for CI and to avoid
duplication when we re-run jobs.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
LEGACY_INCLUDE_PATH has been defaulting to "n" and marked as deprecated
in both v3.2 and v3.3. Drop the option entirely for v3.4.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Initialize 'use_system_gcov', so it is always set before being referenced
in "elif" statement.
Fixes: 1440b19bde ("twister: restore logic to determine which gcov we
should use")
Signed-off-by: Marcin Niestroj <m.niestroj@emb.dev>
Add a new UF2 runner, supporting only the flash capability.
Searches for FAT partitions containing `INFO_UF2.TXT` files,
and can optionally filter on a matching `Board-ID` value in
that file.
Signed-off-by: Peter Johanson <peter@peterjohanson.com>
Stop counting errors as failures and report them as is. This fixes
issues with counting when retrying failures and errors.
We have been adding errors to failures and then double count them when
displaying information on the screen instead of keep an accurate count
in the counter class.
This is now fixed and we do report errors seperately, ie. build errors.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This commit adds access to the string values without a quotes.
Signed-off-by: Radosław Koppel <r.koppel@k-el.com>
Co-authored-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
We do not want to use gcov from the SDK for native posix and unit tests,
instead we want the system gcov. Restore logic that determines that.
Fixes#54226
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The YAML parsing will currently return a None-type if no input data
is empty, and the subsequent check for a substring will cause an
exception.
Signed-off-by: Allan Norgaard <allannk@gmail.com>
Move extra warning option from generic twister script into
compiler-dependent config files.
ARCMWDT compiler doesn't support extra warning options ex.
"-Wl,--fatal-warnings". To avoid build fails flag
"disable_warnings_as_errors" should be passed to twister.
This allows all warning messages and make atomatic test useles.
Signed-off-by: Nikolay Agishev <agishev@synopsys.com>
Make the error messages which appear in CI for invalid commit messages
match the text in the documentation which describes the rules for
commit messages exactly. This hopefully makes it easier for people to
read failing CI results and map them to documentation links describing
what to do instead.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
This commit fixes#54492 where return code 1 is returned
when twister tests fail when twister is called directly and
return code 0 in similar scenario but twister is invoked using
west.
Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
Fix the spelling of Babeltrace library in the warning message displayed
when such a library can not be imported.
Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
Tests under quarantine are not treated as errors in the integration
mode. However --quarantine-verify argument allowing to execute just
test under quarantine and skipping others was not considered in
relation to integration mode. Those skips are wrongly threated as
errors. This commit fix this relation and makes thoses skips not
turned to errors.
fixes: #54516
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
Take an instance that has failed and not whatever was last processed in
the for loop.
Also, change instructions to use `west twister` rather than
`./scripts/twister` to be consistent.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This should fix issue like #54337 where architecture code changes were
filtering out kernel tests, where those changes are being tested. Kernel
and arch code is tightly coupled and we need to run kernel tests.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This is a partial revert of commit
2cee5ff519
("scripts: west_commands: runners: remove deprecated options").
I remarked at the time that the removal of the older way of writing
things from the runners themselves seemed gratuitous since they are
easy to continue to support indefinitely and people may have been used
to the old way of doing things. It didn't seem worth the fight to push
for a revert at the time, though.
Since then I've run into real problems that their removal has caused
in the wild and I am convinced that this part of that patch was wrong.
Restore the original, undeprecated forms of these options, but make it
clear in the command line help that they're just obsolete alternative
spellings at this point.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Currently we just pass in case of the unsupported jtag adapter
is selected. Fix that by rising ValueError as we do in case of
other incorrect arguments.
While I'm at it add few missing test-cases for mdb west runners.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Currently, in case of multi-core target or in case jtag adapter
auto-selectio, we pass unexpected empty argument (empty quotes `''`)
to MDB binary due to argument generation error.
Fix that.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Remove code which looking for `cld` process pid - this functionality
isn't use anymore by twister. And as manual MDB termination is also
fixed now this functionality isn't required for anything now.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Currently we try to start MDB (MetaWare Debugger) in background
with ignored sigint no matter if we are planing to do debug
session (via `west debug`) or just run something on HW or on
nSIM (via `west flash`)
That cause really bad user experience as after we do
`west debug` or `west flash` we can't terminate the debugger
from the console. Moreover even if we terminate the debugger
process from separate terminal the console stil would be
broken so we need to do `stty sane` in it to make it usable
again.
Fix all that by proper starting the debugger process.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Change the check_compliance main invocation to pass an explicit argv
list from main to parse_args(). This does not change anything for direct
invocation, but allows wrapping the compliance script and using it as a
library while controlling its behavior by passing the intended flags to
main().
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
If errors are found and tests are failing, print a brief summary showing
the top 10 items failing with instructions on how to reproduce either
with twister or west.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This will allow something like this:
west build -b qemu_x86 -T samples/hello_world/sample.basic.helloworld
instead of:
cd samples/hello_world
west build -b qemu_x86 -T sample.basic.helloworld
Using this option will allow to take twister output and just pass it to
west to reproduce failures.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Existence of enabled GPIO hog nodes cannot be determined using any of the
existing kconfig functions. Add custom kconfig helper function to determine
whether any GPIO hogs nodes are enabled.
Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
