Commit Graph

2972 Commits

Author SHA1 Message Date
Jian Kang
8c9b06ad82 board: cavs15: Add a option to control signing ways
Zephyr testcases(not SOF case) not use kernel DSP driver to load image
on ADSP board, thus do not need signing with xman. So add a input
'--no-manifest' to specify signing without xman in image. If use DSP
driver load image, we should not specify this.

Signed-off-by: Jian Kang <jianx.kang@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-02-11 13:52:30 -05:00
Andrzej Głąbek
f9c1f89109 scripts: kconfig: Fix uses of edt.compat2okay in helper functions
Check if a given 'compat' is a key in edt.compat2okay before accessing
its entry in dt_compat_on_bus() and dt_nodelabel_has_compat(), as for
a non-existing key a new entry with an empty list would be created,
and dt_compat_enabled() would then incorrectly return "y" when called
for the same 'compat'.

Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
2021-02-09 13:33:10 -05:00
Martí Bolívar
16a7465d93 scripts: zcmake: fix for prerelease cmake versions
Handle a semantic versioning case used by Kitware that is not
supported by the packaging module we're using for version comparisons.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-02-09 09:14:37 -05:00
Maciej Perkowski
b6a69aa820 twister: bugfix: Fix twister output
Fix twister output. "Test suite" used previously did not correspond
to what was counted (test scenarios). A test scenario is an entry from
testcase/sample.yaml under "tests:", which specify configuration used to
build a test. Changes in docs will follow later on. Also replace the
amount of preselectd platforms with the amount of platforms that
were not filtered out and actually tested.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-02-09 06:55:41 -05:00
Martí Bolívar
7adc2a00ea guiconfig: fix search function
The _update_jump_to_matches() function used when searching from the
jump dialog is using the re module without importing it. Fix that.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-02-08 16:24:05 -05:00
Torsten Rasmussen
a088970bc3 scripts: using pathlib for in get_toolchain() method
Followup commit on comment in #32003.

Using pathlib to avoid mixing of `/` and `\` if twister someday will be
able to run on Windows.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-02-05 06:48:18 -05:00
Anas Nashif
ce954d4769 twister: fix HW map fixtures parsing
We were not parsing fixtures from hardware map, so some tests are being
skipped.

Fixes #31948

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-02-05 06:36:14 -05:00
Anas Nashif
2a74053558 twister: fix possible race
Fix possible race when determining available devices. Find available
device and return it in the same operation to avoid the race.

Fixes #31769

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-02-05 06:36:14 -05:00
Torsten Rasmussen
d162e9e1ff cmake: twister: Using common script for toolchain setting verification
Fixes: #30713

Now using the common script `verify-toolchain.cmake` for verifying
toolchain settings.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-02-04 20:05:05 -05:00
Anas Nashif
e8cdbe03d2 twister: remove usage of deprecated gccarmemb variant
gccarmemb was deprecated 2 years ago, so remove it.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-02-04 11:14:44 -05:00
Martí Bolívar
46f2f8cbc9 runners: fix openocd with --use-elf
There are use cases where --use-elf is given and the hex file does not
exist. Handle them.

Fixes: #31944
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-02-03 16:26:27 -05:00
Martí Bolívar
5f327f9edf runners: pyocd: handle None in hex/bin files
Commit 3204554841 (" scripts: runners: error on missing
non-elf outputs") created the possibility of None bin_file and
hex_file attributes in the RunnerConfig without updating pyocd
appropriately. Fix that.

Fixes: #31921
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-02-03 10:36:26 -05:00
Eugeniy Paltsev
afb34e3d5c twister: check if output is connected to tty before 'stty sane' call
As of today we unconditionally call 'stty sane' that leads to
lots of error messages
'stty: standard input: Inappropriate ioctl for device'
when output isn't connected to tty (for example when we launch
twister in Jenkins)

Fix that by check if output is really connected to tty before
'stty sane' call.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
2021-02-02 18:41:12 -05:00
Martí Bolívar
698db69307 west sign: prefer 'python imgtool.py' on windows
We can't trust that a python file is executable on Windows, regardless
of what the mode bits say.

When we find that imgtool is a .py file, run 'python imgtool.py'
instead of 'imgtool.py' on that platform.

Fixes: #31876
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-02-02 17:59:13 -05:00
Martí Bolívar
b4903d44bf west sign: adjust find_imgtool helper
Have it return a list, which is prepended to the final arguments list
passed to subprocess. No behavioral changes expected, but this enables
making it contain multiple strings instead of just one.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-02-02 17:59:13 -05:00
Martí Bolívar
af5a79f6c1 west build: fix board detection
We can't trust the cache to have a CACHED_BOARD just because it exists.

Fixes: #31800

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-02-02 14:08:57 -05:00
Martí Bolívar
3204554841 scripts: runners: error on missing non-elf outputs
The RunnerConfig class stores the locations of the Zephyr output files
in various formats (elf, hex, bin). A longstanding issue with the
representation is that these might not exist if the corresponding
Kconfig options are not set. For example, if
CONFIG_BUILD_OUTPUT_BIN=n, there is no .bin file.

Change this so the type system knows these are Optional[str], not str.

Fix the runners that use non-ELF outputs so they check for the
existence of the relevant file before using it, mostly using a new
ZephyrBinaryRunner.ensure_output helper.

I'm not going to bother with checking for the ELF file itself; that's
always there as far as I can tell.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-02-02 14:01:22 -05:00
Joakim Andersson
97d59a7c6d Bluetooth: host: Mark the option BT_FIXED_PASSKEY as insecure
Mark the option BT_FIXED_PASSKEY as an insecure option.
During Legacy pairing the passkey is used as encryption key, and
brute-forcing this is easy.
During LE Secure Connections the passkey is checked one bit at a time,
so when it is fixed the passkey can be deduced with series of pairing
attempts.

Signed-off-by: Joakim Andersson <joakim.andersson@nordicsemi.no>
2021-02-02 11:09:05 -05:00
Torsten Rasmussen
ea2ab69cf5 scripts: compliance.py updated to support KconfigBasic
This commit lets the Kconfig compliance check run from top of current
git repo instead of only executing inside ZEPHYR_BASE.
This extends the usability of the Kconfig compliance check.

Together with this possibility, a KconfigBasic mode has been added which
will execute the same verification for symbols in the Kconfig tree but
will not check for undefined symbols outside Kconfig.

This is needed as running Kconfig undef check outside the tree in
non-Zephyr repos might pickup `CONFIG_` named symbols in other code.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-01-30 08:09:49 -05:00
Maciej Perkowski
941af213d3 twister: Use alternative test sorting when --device-testing
Change the sorting of test instances if --device-testing is used
within subsets. Test instances are sored depending on the context.
For CI runs the execution order is: "platform1-testA, platform1-testB,
..., platform1-testZ, platform2-testA, ...". For hardware tests, were
multiple platforms can run the tests in parallel, it is more efficient
to run in the order: "platform1-testA, platform2-testA, ...,
platform1-testB, platform2-testB, ..."
This can significantly reduce the tests execution time
for setups with multiple different platforms connected.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-01-30 08:00:36 -05:00
Jingru Wang
99b0720303 twister: rename variable p to log_root
The variable p represents the name of a platform, it shouldn't be
reassigned

Signed-off-by: Jingru Wang <jingru@synopsys.com>
2021-01-29 10:20:10 -05:00
Kumar Gala
16567ccce5 ci: add .buildkite/daily.yml to twister ignore list
Changes to .buildkite/daily.yml don't make any sense to run twister
for since issues will only be seen when the daily builds run so add
it to the twister_ignore.txt list.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2021-01-29 10:19:52 -05:00
Torsten Rasmussen
0cafde6354 scripts: modules: support west group feature
West has introduced support for group tags in:
https://github.com/zephyrproject-rtos/west/pull/454

This means that manifest files might start containing groups.
Zephyr itself only requires west>=0.7.2 where groups are not supported
but other Zephyr based projects might start using the group feature.

When using a west version with group support, then only active projects
will be processed as Zephyr modules.

West versions without group support will consider all projects active.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-01-29 08:06:17 -05:00
Tomi Ollila
66d1b3ce10 kconfig: kconfiglib.py: Backup files only
The _save_old() to return early if <filename> is not file (or symlink
to a file).

This is simplest alternative to avoid attempt to rename /dev/null
(which could succeed).

This also keeps fifos (perhaps nonexistent potential usage but
this is nicer).

If <filename> were directory or socket, after shutil.copyfile(),
writing to the file (by caller, _write_config()), would fail.

Fixes #31362

Co-authored-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Signed-off-by: Tomi Ollila <tomi.ollila@iki.fi>
2021-01-28 10:40:24 -05:00
Anas Nashif
fee9af2bda west: rimage: get config dir from command line
If we do not have the configuration path via cmake, then check if the
configuration path was provided on the command line and use it instead,
if both fail, bail out and die.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-26 06:19:34 -05:00
Anas Nashif
b553166a09 west: sign: add new option -D for configurations
rimage signing tool requires the path to the configuration directory.
This can be fetched using cmake cache if we are building with a module
that has rimage integrated like the sof module, however, we should be
able to sign images that were built standalone and without SOF, so
support this new option to provide the location of the configuration
files for the tool.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-26 06:19:34 -05:00
Peter Bigot
f91e9fba51 device: fix potential truncation of DT-derived device names
While using the encoded path to a device tree node guarantees a unique
identifier for the corresponding device there is a limit on the number
of characters of that name that can be captured when looking up a
device by name from user mode, and the path can exceed that limit.

Synthesize a unique name from the node dependency ordinal instead, and
update the gen_defines script to record the name associated with the
full path in the extern declaration.

Add a build-time check that no device is created with a name that
violates the user mode requirement.

Also update the network device DTS helper functions to use the same
inference for dev_name and label that the real one does, since they
bypass the real one.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2021-01-25 16:26:12 -05:00
Anas Nashif
ec47912292 twister: generate json report on demand
Do not generate json report by default. Use --json-report to generate
the file.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-25 09:56:53 -05:00
Peter Bigot
affa7a1c7e Revert "device: add post-process of elf file to manage device handles"
This reverts commit 40d3653758.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2021-01-23 18:01:03 -05:00
Torsten Rasmussen
774103d050 cmake: pristine.cmake requires SOURCE_DIR and BINARY_DIR as arguments
Calling cmake/pristine.cmake now requires SOURCE_DIR and BINARY_DIR as
arguments.

This ensures that pristine.cmake can evaluate if pristine is requested
on in-source builds, and bail out in such case with an error message.

All uses of `pristine.cmake` has been updated to use the new arguments.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-01-23 08:31:13 -05:00
Daniel Leung
d3218ca515 debug: coredump: remove z_ prefix for stuff used outside subsys
This removes the z_ prefix those (functions, enums, etc.) that
are being used outside the coredump subsys. This aligns better
with the naming convention.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-01-21 22:08:59 -05:00
Daniel Leung
1387e7c7c2 scripts: coredump: fix crashing on non-existent memory region
get_mem_region() may return None for a non-existent memory region
so we need to check first before looking at its properties.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-01-21 22:08:59 -05:00
Kumar Gala
589775f14f requirements: Update to pyocd 0.29 for LPC55S69 support
Pyocd 0.29 supports being able to flash newer LPC55S69 as it handles
the debug unlock handshake.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2021-01-21 15:53:22 -05:00
Peter Bigot
40d3653758 device: add post-process of elf file to manage device handles
Following the idiom used for system calls, add script support to read
the initial application binary to identify which devices are defined,
and to use their offset in the device array as their unique handle
rather than the externally-defined ordinal from devicetree.  The
device dependency arrays are updated to use these handles.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2021-01-21 14:49:04 -06:00
Torsten Rasmussen
3d88083bf1 cmake: zephyr modules: sanitize all module name when used as variable
The introduction of Zephyr module glue code in the Zephyr repository
introduces a Kconfig variable in the form of:
`config ZEPHYR_<MODULE_NAME>_MODULE`.

All Kconfig variables go into `autoconf.h`, therefore it is necessary
to sanitize the Kconfig variable, so that it does not contain special
characters. To ensure consistent variable name, then the module name
will be sanitized in all variable use in both Kconfig and CMake.
The sanitization is done be replacing all special characters with an
underscore, `_`.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-01-20 12:59:19 +01:00
Torsten Rasmussen
1bd0b29918 cmake: scripts: supporting gen_relocate_app feature in windows
Fixes: #28847

This commit fixes two places that was causing gen_relocate_app to fail
in windows.

gen_relocate_app.py now splits only on first `:` in
`<MEM_REGION>:<file>`.

Windows contains `:` in path, thus only first `:` is valid for splitting
mem region and file path.

Second part of the issue is fixed in CMake where `'` was used for
quoting of command arguments.
This causes a file not found on Windows because the final `'` would be
treated as part of the filename. Similar the first `'` would be treated
as path of the mem region name.

This is fixed by using `"` for quoting, which works correctly on all
platforms.

gen_relocate_app.py:403: UserWarning: File: .../kernel/sem.c' Not found
Note the stray `'`                                         ^^^

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-01-20 11:51:47 +01:00
Torsten Rasmussen
3361b154ff scripts: check_compliance.py: include known Zephyr Kconfig glue code
Fixes: #31436

This fix adds ZEPHYR_<MODULE_NAME>_KCONFIG variable to compliance check.
This ensures that Zephyr Kconfig glue code is included in the
compliance check, and thus remove the issue reported in #31436.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-01-20 11:21:16 +01:00
Anas Nashif
989ebf6c35 kernel: add vrfy hooks to support userspace with condvar
Add needed vrfy hooks for userspace support.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-19 08:55:47 -05:00
Andrzej Głąbek
02819489a2 scripts: module: Add missing uses of the module name field
This is a follow-up to commit f24f88324c.

Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
2021-01-19 10:17:56 +01:00
Dominik Ermel
b493acedea scripts/requirements: Add required version to imgtool in extras
The minimal version of imgtool has been set at 1.7.1

Signed-off-by: Dominik Ermel <dominik.ermel@nordicsemi.no>
2021-01-18 16:59:29 -05:00
Marc Herbert
38eed65b2a scripts: move pip pykwalify from requirements-build-test to -base
... because zephyr_module needs it.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2021-01-18 16:56:24 -05:00
Torsten Rasmussen
f24f88324c scripts: module: support for name field in zephyr/module.yml
The folder name of a Zephyr module is also used as its module name
when integrating it into the build system.

This means that a Zephyr module, BAR, located in:
<workspace>/modules/foo
                    |--- zephyr
                          |--- CMakeLists.txt
                          |--- Kconfig

will be referred to as FOO in the build system, that is:
ZEPHYR_FOO_MODULE_DIR==<workspace>/modules/foo
ZEPHYR_FOO_CMAKE_DIR==<workspace>/modules/foo/zephyr

The `name` field allows the module to specify its module name,
independent of its location like:

<workspace>/modules/foo/zephyr/module.yml
```
name: bar
build:
  cmake: zephyr
```

will instead be referred to as BAR in the build system, that is:
ZEPHYR_BAR_MODULE_DIR==<workspace>/modules/foo
ZEPHYR_BAR_CMAKE_DIR==<workspace>/modules/foo/zephyr

This allows for greater flexibility of relocating Zephyr modules in
other folders and at the same time be guaranteed that other modules
depending on `ZEPHYR_BAR_MODULE_DIR` is still working.

If `name` field is not specified in `module.yml`, then the existing
behavior of using the folder name will be used.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-01-18 21:53:20 +01:00
Torsten Rasmussen
3673e288bd modules: introducing MODULE_EXT_ROOT to allow glue code in Zephyr repo
This commit introduces MODULE_EXT_ROOT which allows CMake and Kconfig
glue code to be placed outside of the Zephyr module repository.

This allows for placing glue code in Zephyr, but also allows users to
specify custom MODULE_EXT_ROOTs for glue code using either
`-DMODULE_EXT_ROOT` or `zephyr/module.yml` with
`build:settings:module_ext_root` settings.

MODULE_EXT_ROOT' is a list of directories, similar to other roots such
as BOARD_ROOT, DTS_ROOT, etc.
The Zephyr repo folder ${ZEPHYR_BASE} is always to the MODULE_EXT_ROOT
list as lowest priority.
For each MODULE_EXT_ROOT, the file
`<module_ext_root>/modules/modules.cmake` will be processed.

In Zephyr repo, the folder `modules/<module>/` contains CMakeLists.txt
and Kconfig glue code for the Zephyr module.

A Zephyr module can specify that CMakeLists.txt and Kconfig glue code is
placed in an external module root by specifying:
```
build:
  cmake-ext: True
  kconfig-ext: True
```

It is still possible to place the CMakeLists.txt and Kconfig files
directly in the Zephyr module using the existing:
```
build:
  cmake: <path>
  kconfig: <file>
```.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-01-18 18:02:46 +01:00
Anas Nashif
b1080ea036 twister: remove test type from json
This is always the same, so we might as well just remove it for now and
save some space since it is the default anyways. Type can be added later
when it provides more value.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-18 08:30:49 -05:00
Anas Nashif
cd1dccf9bc twister: fix json report
Fix JSON report generation and simplify it a bit.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-18 08:30:49 -05:00
Anas Nashif
211ef41125 twister: debug which coverage tool is being used
Add more debug messages about coverage tool being used.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-18 08:30:49 -05:00
Martí Bolívar
c19c6fb438 Revert "scripts: west build: default build.pristine to auto"
This reverts commit c505ca38cf.

Fixes: #31358

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-01-17 08:58:54 -06:00
Martí Bolívar
f7d33f2988 scripts: dts: gen_defines: don't include descriptions
Before we had a bindings index in the documentation, the generated
header file was (somewhat unfortunately) often our best reference for
what a particular binding or property within a binding ends up doing,
so it made good sense to put the description in the generated file.

Now that we have HTML documentation that's a bit more digestible than
the generated file, though, we can just point users at that. Do that
and remove the inline description from the generated file.

This makes it possible to put C-style multiline comments in the
descriptions themselves, which will be done in subsequent patches.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-01-16 07:59:27 -06:00
Martí Bolívar
00ffc7e7b7 devicetree: add DT_NODE_PATH()
This macro returns a node's full path, given its node identifier.

The entire path to a node is useful information for the user which can
be added to build-time error messages.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-01-16 07:59:27 -06:00
Martí Bolívar
0d5e6c13e9 boards/shields: re-work handling in cmake and west
Remove the boards and shields lists from the 'usage' target output.
That might have been readable at some point long ago in Zephyr's
history, when only a few boards were available, but right now it's
obscuring the high level targets we really want 'usage' to print.

Instead, add 'boards' and 'shields' targets which the user can run to
get those lists, and reference them from the 'usage' output. This
makes 'usage' squintable again. We use the new list_boards.py script
from the 'boards' target.

Reference the 'help' target from 'usage' as well, and drop the
recommendation that people run '--target help' from the 'west build
--help' output for the 'west build --target' option. The canonical
place to look is 'usage' now.

Use the new list_boards.py code from 'west boards' as well, which
allows us to add the board's directory as a format string key, in
addition to its name and architecture.

Keep west-completion.bash up to date. While doing that, I noticed that
a bunch of references to this file refer to a stale location, so fix
those too.

Finally, the 'usage' output is what we print for a failed board or
shield lookup, so that needs to be updated also. Handle that by
invoking boards.cmake and a new shields.cmake in CMake script mode to
print the relevant output.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-01-15 15:07:49 -05:00
Martí Bolívar
8bd6d08b0b scripts: add list_boards.py
This script is essentially a Python rewrite of the CMake code we're
using to print boards in cmake/boards.cmake, plus some extra features.

Having this in Python will simplify some later adjustments to our
'usage' build system target in ways that will make its output easier
to read, while simultaneously making 'west boards' more useful.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-01-15 15:07:49 -05:00
Erwan Gouriou
b711028539 scripts/Kconfig: Add dt_node_has_prop Kconfig function
Add dt_node_has_prop function to query the presence of 'prop'
for given node label.

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2021-01-15 10:29:30 -05:00
Kumar Gala
98b6e4f834 devicetree: generate extern's for devicetree struct devices
Generate a header (device_extern.h) that handles extern of possible
device structs that would come from devicetree.  This removes the need
for DEVICE_DT_DECLARE and DEVICE_DT_INST_DECLARE which we can remove.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2021-01-15 07:16:21 -06:00
Anas Nashif
5d50797dad ci: handle checpatch warnings as errors
Fail CI if we have both errors and warnings.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-14 17:29:37 -05:00
Anas Nashif
827ecb7bd0 twister: fix build_on_all configuration
Logic for scope of platforms to be checked was changed and this options
now only applies to limited scope, fix this and reset scope when a
testcase declares it needs to build on available platforms.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-14 13:59:42 -05:00
Kumar Gala
3befd7d45f cleanup: rename sanitycheck references to twister
Cleanup references to sanitycheck that should now be twister.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2021-01-14 07:50:52 -06:00
Martí Bolívar
7745f3edba west boards: add --name filter
This is an optional regular expression that can be used to filter the
output.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-01-14 12:39:21 +01:00
Marc Herbert
6d36386e13 requirements-compliance: restrict junitparser version < 2
junitparser version 2 is incompatible with check_compliance.py, it fails
like this:

File "./scripts/ci/check_compliance.py", line 295, in parse_kconfig
   self.skip("Not a Zephyr tree (ZEPHYR_BASE unset)")
File "./scripts/ci/check_compliance.py", line 141, in skip
   self.case.result = Skipped(msg, "skipped")
File "/usr/local/lib/python3.9/site-packages/junitparser/junitparser.py"
   line 682, in result
   for entry in value:

TypeError: 'Skipped' object is not iterable

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2021-01-13 07:58:37 -05:00
Marc Herbert
205dcdf636 check_compliance: fix broken format() in exception handler
Likely an accident when commit 288ae28c13 moved the script to the main
repo. Stayed unnoticed because this handler rarely ever runs.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2021-01-13 07:58:37 -05:00
Guennadi Liakhovetski
3de40b48c3 rimage: update rimage: add configuration and extended manifest
rimage dropped its "-m" parameter and switched over to using "-c"
for a configuration file, including a target name.

Add support for extended manifest for all cAVS versions.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
2021-01-11 16:10:23 -05:00
Marc Herbert
c6067e7f1c requirements-doc: add missing pkwalify
doc/CMakeLists.txt uses zephyr_module

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2021-01-11 14:51:45 -05:00
Hake Huang
d5c60ff2f1 test_version: update tset version with 12 digits
update the version with 12 digits in aligned with uniq request from

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
2021-01-10 17:00:17 -05:00
Eugeniy Paltsev
79b3a50772 twister: BinaryHandler: call try_kill_process_by_pid in a 'with' scope
The commit
531fe89e80 (sanitycheck: use multiprcoessing instead of threads)
introduce regression for ARC nsim_hs_smp platform.

The regression is that verification on nsim_hs_smp hangs. That
happens because now we don't call try_kill_process_by_pid()
in the 'with' scope of the 'subprocess.Popen' when we spawning
BinaryHandler thread. Previously it was called via `terminate`
method of `BinaryHandler` but it was changed in 531fe89e80.
So if we can't terminate the simulator in a normal way
(which is expected for 'mdb' which is used for nsim_hs_smp
simulation) we will hang forever - we will never return from
'with' scope of the 'subprocess.Popen' as we are waiting for
process termination but the try_kill_process_by_pid() is located
latter and we never reach it.

Fix that.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
2021-01-10 15:42:48 -05:00
Anas Nashif
b2a5782570 ci: remove west extension call
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-08 07:49:19 -05:00
Johan Hedberg
4ba387829b edtlib: Match any parent bus when binding lacks an explicit on-bus
There are some drivers in the tree that support devices on multiple
different buses, although so far this has not been represented in
device tree using the bus concept. In order to convert these drivers &
bindings to refer to a formal bus in device tree we need to be able to
match bindings which lack an explicit "on-bus: ..." value against any
parent bus. This will also be needed for any external bindings, since
those would not be aware of on-bus (as it's a Zephyhr-specific
extension).

The two drivers I'm particularly targeting is the ns16550 UART driver
(drivers/serial/uart_ns16550.c) and the DW I2C driver
(drivers/i2c/i2c_dw.c). They both support devices with a fixed MMIO
address as well as devices connected and discovered over PCIe. The
only issue is that instead of encoding the bus information the proper
DT way these bindings use a special "pcie" property in the DT node
entries to indicate whether the node is on the PCIe bus or not.

Being able to convert the above two drivers to use the DT bus concept
allow the removal of "hacks" like this:

 if DT_INST_PROP(0, pcie) || \
       DT_INST_PROP(1, pcie) || \
       DT_INST_PROP(2, pcie) || \
       DT_INST_PROP(3, pcie)

to the more intuitive:

 if DT_ANY_INST_ON_BUS_STATUS_OKAY(pcie)

This also has the benefit that the driver doesn't need to make any
arbitrary assumptions of how many matching devices there may be but
works for any number of matches. This is already a problem now since
e.g. the ns16550 driver assumes a maximum of 4 nodes, whereas
dts/x86/elkhart_lake.dtsi defines up to 9 different ns16550 nodes.

Signed-off-by: Johan Hedberg <johan.hedberg@intel.com>
2021-01-07 20:07:12 +02:00
Anas Nashif
f1dea627cc ci: fetch sof submodules
Fetch submodules needed to build with SOF.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-06 07:53:46 -06:00
Martí Bolívar
a8612eb308 runners: fix nrfjprog UICR check
It's fine to flash an image with UICR data as long as the user has
given any of the --erase, --recover, or --force options. Silence the
error correctly.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-01-05 08:44:05 -06:00
Jingru Wang
b9c953c126 twister: add 12 hexadecimal digits to version
With the earlier version of Git, git describe command will
use the default 7 hexadecimal digits as the abbreviated
object name. But currently the Linux kernel project needs
11 to 12 hexdigits to stay unique.So it's necessary to add
--abbrev=12 to ensure twister always generate the same tags.

In https://testing.zephyrproject.org/daily_tests/versions.json,
all tags have 10 hexadecimal digits.

Signed-off-by: Jingru Wang <jingru@synopsys.com>
2021-01-04 08:42:40 -05:00
Fabio Utzig
7cc84af393 doc: fix broken text search
Require Sphinx<3.4.0 to avoid the issue described in:

https://github.com/sphinx-doc/sphinx/issues/8603

This requirement can be relaxed once a new sphinx-rtd-theme is
released.

Signed-off-by: Fabio Utzig <fabio.utzig@nordicsemi.no>
2021-01-04 07:57:07 -05:00
Eugeniy Paltsev
3134bc1ea0 runners: nsim: allow to use runner if gdb is missing
As of today we check for gdb presence in 'do_create' method and
raise an exception in case of gdb is missing. That makes nsim
runner unusable without gdb even for the commands which don't use
it (like 'flash' command).

Fix that.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
2020-12-27 18:19:10 +01:00
Martí Bolívar
38ede5aded dts: handle unspecified phandle-array elements
The DTS language permits zeroing out phandles in a phandle array to
say "there's nothing at this index", and dtlib manages that correctly,
but edtlib and gen_defines.py aren't equipped to do so.

Fix this by allowing None elements in the lists of ControllerAndData
values returned by edtlib for such properties.

Handle that in gen_defines.py by setting the generated
DT_N_<node>_P_<prop>_IDX_<i>_EXISTS macro to 0 in such cases.

The DT_N_<node>_P_<prop>_LEN macro still accounts for the entire
length of the phandle-array; it's just that some indexes may be
missing data.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-22 07:34:29 -06:00
Martí Bolívar
42504e8c2c dts: tests: delete extraneous whitespace
This line is required by the commit linter.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-22 07:34:29 -06:00
Anas Nashif
4fa82c44af twister: fix hardware map load/save
Do not store shared memory variables into yaml and check for validity of
imported hardware map before attempting to modify it.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-19 19:20:09 -05:00
Anas Nashif
5be8afcf73 ci: update versions file with weekly label
Sync with version in test_results.
Set the version to be a weekly on Mondays.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-18 07:46:47 -05:00
Jingru Wang
f20c15caaf sanitycheck: Add exception check
Add exception check for psutil when running CI with nSIM

Sometimes nSIM random failures may turn nsimdrv process into zombie
process, parent process can't terminate it by calling proc.wait(0.1)
and kill command doesn't apply to zombie process

Signed-off-by: Jingru Wang <jingru@synopsys.com>
2020-12-17 09:18:00 -05:00
Jacob Trueb
278aca926b scripts: west_commands: runners: Extend jlink timeout
With Rosetta, jlink command runs longer than 0.1 seconds.

Signed-off-by: Jacob Trueb <jtrueb@northwestern.edu>
2020-12-16 14:26:13 +01:00
Anas Nashif
4594370b46 twister: remove some leftover sanitycheck mentions
Few leftover mentions of Sanitycheck.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-14 14:13:16 -05:00
Martí Bolívar
f673e2060c edtlib: warn on non-tokenizable enums
These are likely enough defined by mistake to emit a warning for.

Adjust tests to match, tweaking the test_warnings() setup: now that
we've got several test cases, it's a bit cleaner not to have to
copy/paste the ('edtlib', WARNING, ...) part of every expected log
record tuple.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-13 19:25:24 -05:00
Martí Bolívar
0985849aa2 edtlib: use standard logging module
The edtlib strategy for emitting warnings is to print directly to
standard error. This in turn requires hacks to drop stored references
to stderr in various _warn_file attributes so the EDT objects can be
pickled.

In general, I think it's not really appropriate for library modules
like edtlib to be printing to stderr directly. The user should be able
to configure logging for general utility data munging modules like
this as they please, and not just deciding what file to print to.

Move this around so the standard logging module is used instead. We
can preserve backwards compatibility in gen_defines by customizing the
'edtlib' logging module behavior so it prints the exact same thing it
always has.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-13 19:25:24 -05:00
Peter Bigot
345da78b5b devicetree: basic support for enum tokens
Whenever a devicetree binding defines a string property whose
enumerated values are all tokenizable, generate C macros for each
property value that are the corresponding tokens.

Note that "token" is distinct from "identifier": both 'foo' and '123'
are valid tokens, but only 'foo' is a valid identifier. We permit some
strings which are not valid identifiers in anticipation that the
generalization may be useful, e.g. when defining macros that paste the
token onto a prefix that makes the whole thing an identifier.

Fixes: #21273
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-13 19:25:24 -05:00
Martí Bolívar
b6dc0a24f7 edtlib: improve enum support
Add enum_tokenizable and enum_upper_tokenizable to PropertySpec. These
allow a PropertySpec to declare that it both has an enumeration of
values and all of them are strings which are "tokenizable". Don't
bother extending Property with these; the user can access the
information through Property.spec now, so the extra delegation is
unnecessary.

See the docstrings for details on what "tokenizable" means. The basic
idea is that we should be able to use the DT binding's enum values as
C 'enum' enumerators in a "reasonable way".

Add val_as_token to Property. This produces a canonical token for the
property value.

Add tests for this feature in particular and property enumerations in
general.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-13 19:25:24 -05:00
Martí Bolívar
88094ecfb1 edtlib: tweak Property initialization
In the 'normal' case of a property whose definition is taken from a
binding YAML file, a fair number (three at present) of the attributes
available on Property objects are directly taken from the
corresponding PropertySpec object.

Refactor the internals of how a Property gets initialized so that it
has a direct reference to its PropertySpec, and make those attributes
properties which just delegate to the PropertySpec (which in turn just
delegate to the binding). Additionally, expose the PropertySpec
directly.

This will make it easier to extend the Property class with additional
attributes that normally come from the PropertySpec without having to
touch all the locations where Property.__init__ is called.

In the case of the 'default' properties, we handle this by dummying
out some PropertySpec objects. These dummy PropertySpecs in turn
require a dummy Binding.

This change has the advantage that it improves the degree to which
these defaults are checked, e.g. it makes sure that 'status' is one of
the permitted values.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-13 19:25:24 -05:00
Anas Nashif
b517b1fc60 sanitycheck: add more logging
This will let us know what is going on, especially on operations that
take some time to complete.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 17:41:40 -05:00
Anas Nashif
2d4891725c sanitycheck: handle keyboard interrupts
Handle keyboard interrupts (Ctrl-C) and terminate gracefully.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 17:41:40 -05:00
Anas Nashif
a5d16ab027 sanitycheck: add option to store platform reports
Now we create plaform reports for each enabled platform, this is time
consuming, especially if you are not using those reports for anything.

Make this an option and enable only if the generated reports are going
to be used.

Individual reports can also be generated from the JSON file if needed.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 17:41:40 -05:00
Anas Nashif
7822e7b50d sanitycheck: retry-failed comment
Add a comment about how deal with retries when a test fails to build.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 17:41:40 -05:00
Martí Bolívar
8fce3004fe runners: mdb: fix imports
Runner implementations are only allowed to unconditionally import
modules in the python standard library. They are not allowed to import
anything that comes from pip or other third party sources unless they
catch the ImportError and gracefully do nothing.

Fix the imports in the mdb runner accordingly, sorting the imports
into sections to make it clearer what's stdlib, what's runners, and
what's third party while we're here.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-11 16:49:59 -05:00
Anas Nashif
487af47f58 twister: add a wrapper script with old name
Add a simpel wrapper script called 'sanitycheck' to be removed after
Zephyr 2.5.

This script has a warning and a 5 sec delay to get the attention of the
caller.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
5471398ff0 modules: rename options for twister support
Zephyr modules with test support should also be using twister instead of
sanitycheck.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
5725ed8dd1 twister: fixed pylint issues
Fixed import and unused variable warning.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
9318e6c8f7 twister: tests: rename sanitycheck -> twister
Change all tests to use twister instead of sanitycheck.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
a05d056943 ci: rename sanitycheck -> twister
Change all CI scripts to use the new script name: twister.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
2e4b0fad98 twister: adapt platform schema
Adapt schema to use twister instead of sanitycheck.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
92c9d4ba14 twister: move unit tests
Move all unittests to a new folder with the twister name.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
b18f311b3e twister: change all occurances of sanitycheck
mass search/replace for sanitycheck replacing it with the new name:
twiser.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
2d351e7ffc sanitycheck: rename main script
We are now known as twister.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
e508babfed sanitycheck: move sanity_chk -> pylib/twister
Move the "module" classes used by sanitycheck into a new location under
pylib, where we can have other python libraries reside instead of having
them all over the place.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
303fc90311 sanitycheck: remove obsolete .gitignore
Remove obsolete .gitignore file

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
94f6826d28 sanitycheck: move release data to release/
Move the CSV files with release related data out of the python directory
into scripts/releases, which is more appropriate.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
c24bf6f183 sanitycheck: move schemas to scripts/schemas
Move yaml schemas into one single location. We have quite a few schema
used across the tree, one common place will make them easier to find and
reference.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00