Commit Graph

3092 Commits

Author SHA1 Message Date
James Harris
074dbb9982 scripts: gen_syscalls: declare syscalls with void instead of zero args
In C, `void foo(void);` and `void foo();` mean subtly different things.
The former means "foo takes zero arguments"; the latter means
"foo takes an unspecified number of arguments". This can result in
calling convention mismatches in exceptional cases.

Change to emitting `(void)` instead of `()` for a syscall with
zero arguments.

Signed-off-by: James Harris <james.harris@intel.com>
2021-03-05 12:42:36 -05:00
Andy Ross
098fce351f scripts/twister: Fix race with device-testing
CPython is sometimes described as "single threaded" due to the GIL,
but the interpreter will still "preemptively" switch between threads
(the details seem poorly documented).

So the time between checking whether acquire is 1 and decrementing the
count could result in more than one thread seeing an "available"
device, and more than one test being run (simultaneously, on the same
physical device!).  We have a big herd of threads all polling for
this, so in a large test run this would happen maybe one time out of
20-30 tries.

Use a lock.  Also remove the very similar looking
DUT.get_available_device() method, which had the same bug but appears
to be dead code.

Fixes #32679

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-03-04 12:39:54 -05:00
Anas Nashif
82daea910b ci: limit CI to only changed architecture
If files are changed for architecture code, then only run CI on those
architectures.

Combine all scripts into one and make it simpler.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-03-04 07:14:11 -06:00
Anas Nashif
28bc79b49f ci: do not run twister CI when changing checkpatch
changes to checkpatch have no impact on code and twister results.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-03-04 07:14:11 -06:00
Anas Nashif
3572b518f6 ci: remove shippable references
We now use buildkite.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-03-04 07:14:11 -06:00
Joakim Andersson
200f4687e0 Bluetooth: host: Log security keys needed by sniffer
Log the security keys that the sniffer needs in order to sucessfully
decrypt the connection.
This option allows the sniffer to work in the cases where enabling
using the SMP debug keys is not wanted, either because it changes the
way the peer behaves or is denied by the peer.
It also enables the sniffer to decrypt a connection where the bond
already exists.

Signed-off-by: Joakim Andersson <joakim.andersson@nordicsemi.no>
2021-03-04 14:20:25 +02:00
Martí Bolívar
fbd34dca1e twisterlib: add deprecated DT props to warnings_as_errors
Use EXTRA_GEN_DEFINES_ARGS to error out on deprecated devicetree
properties when warnings are treated as errors.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-03-03 10:22:36 -06:00
Martí Bolívar
df5a55c638 dts: gen_defines: add CLI option to error out on deprecated props
Allow users to set the new EDT constructor argument which errors out
on deprecated properties via a command line argument.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-03-03 10:22:36 -06:00
Martí Bolívar
40c4f0048e dts: edtlib: add EDT kwarg to error out on deprecated props
We'd like to start eliminating deprecated properties from upstream
Zephyr devicetrees. To make that possible in the build system, add an
EDT kwarg that does just that.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-03-03 10:22:36 -06:00
Martí Bolívar
563a46c7d4 dts: clean up binding syntax checks
Commit 6bf761fc0a ("dts: Remove support
for deprecated DTS binding syntax") removed most of the support for
the 'legacy' bindings syntax.

A few straggler keys are still around in the bindings check code,
though. This allows some legacy keys which should cause errors to pass
silently instead.

Fix the error handling and print good errors for cases which are
removed, just in case someone is still using them somewhere.

Clean up some other error messages in the same function while we're
here.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-03-03 08:18:10 -06:00
Carlo Caione
6ff37886dd aarch64: mem_domain: Introduce k_mem_partition_attr_t
Introduce the kernel macros for memory attribution also for AArch64.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2021-03-01 09:25:44 -05:00
Kumar Gala
4aac908973 devicetree: add DT_NODE_FULL_NAME()
This macro returns a node's name with unit-adddress, given its node
identifier.

The node name is useful information for the user to utilize for debug
information, similar to DT_NODE_PATH, or DT_LABEL.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2021-02-25 19:24:08 -06:00
Kumar Gala
270a05f697 dts: gen_defines: Change f-string to normal string
Change the use of a f-string to a normal string as there is nothing
that needs formatting in the particular instance.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2021-02-25 19:24:08 -06:00
Jordan Yates
dcaabb860f west: runners: jlink: prefer .hex over .bin
Update the jlink runner to prefer flashing .hex files instead of .bin.
This can increase programming speed in cases where there are large
amounts of padding in an application.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2021-02-25 10:59:13 -06:00
peng1 chen
6478735493 twister: fix the inconsistent total tests number
sometimes a test instance hasn't been boot up to execute successfully,
so the results is an empty dictionary, it will not be added into final
twister_report.xml file.
in order to include it, fill the results as BLOCK.

Signed-off-by: peng1 chen <peng1.chen@intel.com>
2021-02-24 16:00:35 -05:00
Pete Johanson
310a464acf cmake: Add UF2 as an additional build output option.
Add ability to build a UF2 (https://github.com/Microsoft/uf2)
image as an additional output type. This leverages the code
partition offset for the UF2 base address, and a configurable
UF2 family ID.

Includes an unmodified (except for headers for licensing, pylit
disabling) version of the uf2conv.py script copied the UF2
format specification repository, used to convert the bin to UF2.

Origin: UF2 file format specification reference utilies
License: MIT
URL: https://github.com/microsoft/uf2/blob/master/utils/uf2conv.py
commit: 587abb8b909266e9b468d6284f2fbd425235d1b5
Signed-off-by: Pete Johanson <peter@peterjohanson.com>
2021-02-23 21:11:04 +01:00
Sebastian Wezel
148dbdc9fe twisterlib: checking for classname and testname
Currently duplicated testcases are only determined by the classname.
This results in testcases not beeing added to the twister.xml if the
testcases are from the same class (e.g. sample.yaml), but with different
names.

The changed check if the combination of classname and testname
already exists.

Signed-off-by: Sebastian Wezel <sebastian.wezel@nordicsemi.no>
2021-02-23 10:57:14 -05:00
Peter Bigot
d554d34137 device: add post-process of elf file to manage device handles
Following the idiom used for system calls, add script support to read
the initial application binary to identify which devices are defined,
and to use their offset in the device array as their unique handle
rather than the externally-defined ordinal from devicetree.  The
device dependency arrays are updated to use these handles.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2021-02-19 15:46:16 -05:00
Flavio Ceolin
155a06a1c9 scripts: Update paths on tags.sh
Update source paths.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-02-19 05:28:09 -06:00
Andrei Emeltchenko
10fa48dd3e twister: Include coverage support for device testing
Include coverage support for device testing. It works by switching
capture_coverage to True (as it would meet coverage data start
pattern). Then we continue tot read coverage data until we get
coverage data end pattern. Otherwise, after receiving test result
pattern, we close serial console and do not get time to find start
coverage data pattern.

Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
2021-02-15 08:13:48 -05:00
Jian Kang
8c9b06ad82 board: cavs15: Add a option to control signing ways
Zephyr testcases(not SOF case) not use kernel DSP driver to load image
on ADSP board, thus do not need signing with xman. So add a input
'--no-manifest' to specify signing without xman in image. If use DSP
driver load image, we should not specify this.

Signed-off-by: Jian Kang <jianx.kang@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-02-11 13:52:30 -05:00
Andrzej Głąbek
f9c1f89109 scripts: kconfig: Fix uses of edt.compat2okay in helper functions
Check if a given 'compat' is a key in edt.compat2okay before accessing
its entry in dt_compat_on_bus() and dt_nodelabel_has_compat(), as for
a non-existing key a new entry with an empty list would be created,
and dt_compat_enabled() would then incorrectly return "y" when called
for the same 'compat'.

Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
2021-02-09 13:33:10 -05:00
Martí Bolívar
16a7465d93 scripts: zcmake: fix for prerelease cmake versions
Handle a semantic versioning case used by Kitware that is not
supported by the packaging module we're using for version comparisons.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-02-09 09:14:37 -05:00
Maciej Perkowski
b6a69aa820 twister: bugfix: Fix twister output
Fix twister output. "Test suite" used previously did not correspond
to what was counted (test scenarios). A test scenario is an entry from
testcase/sample.yaml under "tests:", which specify configuration used to
build a test. Changes in docs will follow later on. Also replace the
amount of preselectd platforms with the amount of platforms that
were not filtered out and actually tested.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-02-09 06:55:41 -05:00
Martí Bolívar
7adc2a00ea guiconfig: fix search function
The _update_jump_to_matches() function used when searching from the
jump dialog is using the re module without importing it. Fix that.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-02-08 16:24:05 -05:00
Torsten Rasmussen
a088970bc3 scripts: using pathlib for in get_toolchain() method
Followup commit on comment in #32003.

Using pathlib to avoid mixing of `/` and `\` if twister someday will be
able to run on Windows.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-02-05 06:48:18 -05:00
Anas Nashif
ce954d4769 twister: fix HW map fixtures parsing
We were not parsing fixtures from hardware map, so some tests are being
skipped.

Fixes #31948

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-02-05 06:36:14 -05:00
Anas Nashif
2a74053558 twister: fix possible race
Fix possible race when determining available devices. Find available
device and return it in the same operation to avoid the race.

Fixes #31769

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-02-05 06:36:14 -05:00
Torsten Rasmussen
d162e9e1ff cmake: twister: Using common script for toolchain setting verification
Fixes: #30713

Now using the common script `verify-toolchain.cmake` for verifying
toolchain settings.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-02-04 20:05:05 -05:00
Anas Nashif
e8cdbe03d2 twister: remove usage of deprecated gccarmemb variant
gccarmemb was deprecated 2 years ago, so remove it.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-02-04 11:14:44 -05:00
Martí Bolívar
46f2f8cbc9 runners: fix openocd with --use-elf
There are use cases where --use-elf is given and the hex file does not
exist. Handle them.

Fixes: #31944
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-02-03 16:26:27 -05:00
Martí Bolívar
5f327f9edf runners: pyocd: handle None in hex/bin files
Commit 3204554841 (" scripts: runners: error on missing
non-elf outputs") created the possibility of None bin_file and
hex_file attributes in the RunnerConfig without updating pyocd
appropriately. Fix that.

Fixes: #31921
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-02-03 10:36:26 -05:00
Eugeniy Paltsev
afb34e3d5c twister: check if output is connected to tty before 'stty sane' call
As of today we unconditionally call 'stty sane' that leads to
lots of error messages
'stty: standard input: Inappropriate ioctl for device'
when output isn't connected to tty (for example when we launch
twister in Jenkins)

Fix that by check if output is really connected to tty before
'stty sane' call.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
2021-02-02 18:41:12 -05:00
Martí Bolívar
698db69307 west sign: prefer 'python imgtool.py' on windows
We can't trust that a python file is executable on Windows, regardless
of what the mode bits say.

When we find that imgtool is a .py file, run 'python imgtool.py'
instead of 'imgtool.py' on that platform.

Fixes: #31876
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-02-02 17:59:13 -05:00
Martí Bolívar
b4903d44bf west sign: adjust find_imgtool helper
Have it return a list, which is prepended to the final arguments list
passed to subprocess. No behavioral changes expected, but this enables
making it contain multiple strings instead of just one.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-02-02 17:59:13 -05:00
Martí Bolívar
af5a79f6c1 west build: fix board detection
We can't trust the cache to have a CACHED_BOARD just because it exists.

Fixes: #31800

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-02-02 14:08:57 -05:00
Martí Bolívar
3204554841 scripts: runners: error on missing non-elf outputs
The RunnerConfig class stores the locations of the Zephyr output files
in various formats (elf, hex, bin). A longstanding issue with the
representation is that these might not exist if the corresponding
Kconfig options are not set. For example, if
CONFIG_BUILD_OUTPUT_BIN=n, there is no .bin file.

Change this so the type system knows these are Optional[str], not str.

Fix the runners that use non-ELF outputs so they check for the
existence of the relevant file before using it, mostly using a new
ZephyrBinaryRunner.ensure_output helper.

I'm not going to bother with checking for the ELF file itself; that's
always there as far as I can tell.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-02-02 14:01:22 -05:00
Joakim Andersson
97d59a7c6d Bluetooth: host: Mark the option BT_FIXED_PASSKEY as insecure
Mark the option BT_FIXED_PASSKEY as an insecure option.
During Legacy pairing the passkey is used as encryption key, and
brute-forcing this is easy.
During LE Secure Connections the passkey is checked one bit at a time,
so when it is fixed the passkey can be deduced with series of pairing
attempts.

Signed-off-by: Joakim Andersson <joakim.andersson@nordicsemi.no>
2021-02-02 11:09:05 -05:00
Torsten Rasmussen
ea2ab69cf5 scripts: compliance.py updated to support KconfigBasic
This commit lets the Kconfig compliance check run from top of current
git repo instead of only executing inside ZEPHYR_BASE.
This extends the usability of the Kconfig compliance check.

Together with this possibility, a KconfigBasic mode has been added which
will execute the same verification for symbols in the Kconfig tree but
will not check for undefined symbols outside Kconfig.

This is needed as running Kconfig undef check outside the tree in
non-Zephyr repos might pickup `CONFIG_` named symbols in other code.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-01-30 08:09:49 -05:00
Maciej Perkowski
941af213d3 twister: Use alternative test sorting when --device-testing
Change the sorting of test instances if --device-testing is used
within subsets. Test instances are sored depending on the context.
For CI runs the execution order is: "platform1-testA, platform1-testB,
..., platform1-testZ, platform2-testA, ...". For hardware tests, were
multiple platforms can run the tests in parallel, it is more efficient
to run in the order: "platform1-testA, platform2-testA, ...,
platform1-testB, platform2-testB, ..."
This can significantly reduce the tests execution time
for setups with multiple different platforms connected.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-01-30 08:00:36 -05:00
Jingru Wang
99b0720303 twister: rename variable p to log_root
The variable p represents the name of a platform, it shouldn't be
reassigned

Signed-off-by: Jingru Wang <jingru@synopsys.com>
2021-01-29 10:20:10 -05:00
Kumar Gala
16567ccce5 ci: add .buildkite/daily.yml to twister ignore list
Changes to .buildkite/daily.yml don't make any sense to run twister
for since issues will only be seen when the daily builds run so add
it to the twister_ignore.txt list.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2021-01-29 10:19:52 -05:00
Torsten Rasmussen
0cafde6354 scripts: modules: support west group feature
West has introduced support for group tags in:
https://github.com/zephyrproject-rtos/west/pull/454

This means that manifest files might start containing groups.
Zephyr itself only requires west>=0.7.2 where groups are not supported
but other Zephyr based projects might start using the group feature.

When using a west version with group support, then only active projects
will be processed as Zephyr modules.

West versions without group support will consider all projects active.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-01-29 08:06:17 -05:00
Tomi Ollila
66d1b3ce10 kconfig: kconfiglib.py: Backup files only
The _save_old() to return early if <filename> is not file (or symlink
to a file).

This is simplest alternative to avoid attempt to rename /dev/null
(which could succeed).

This also keeps fifos (perhaps nonexistent potential usage but
this is nicer).

If <filename> were directory or socket, after shutil.copyfile(),
writing to the file (by caller, _write_config()), would fail.

Fixes #31362

Co-authored-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Signed-off-by: Tomi Ollila <tomi.ollila@iki.fi>
2021-01-28 10:40:24 -05:00
Anas Nashif
fee9af2bda west: rimage: get config dir from command line
If we do not have the configuration path via cmake, then check if the
configuration path was provided on the command line and use it instead,
if both fail, bail out and die.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-26 06:19:34 -05:00
Anas Nashif
b553166a09 west: sign: add new option -D for configurations
rimage signing tool requires the path to the configuration directory.
This can be fetched using cmake cache if we are building with a module
that has rimage integrated like the sof module, however, we should be
able to sign images that were built standalone and without SOF, so
support this new option to provide the location of the configuration
files for the tool.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-26 06:19:34 -05:00
Peter Bigot
f91e9fba51 device: fix potential truncation of DT-derived device names
While using the encoded path to a device tree node guarantees a unique
identifier for the corresponding device there is a limit on the number
of characters of that name that can be captured when looking up a
device by name from user mode, and the path can exceed that limit.

Synthesize a unique name from the node dependency ordinal instead, and
update the gen_defines script to record the name associated with the
full path in the extern declaration.

Add a build-time check that no device is created with a name that
violates the user mode requirement.

Also update the network device DTS helper functions to use the same
inference for dev_name and label that the real one does, since they
bypass the real one.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2021-01-25 16:26:12 -05:00
Anas Nashif
ec47912292 twister: generate json report on demand
Do not generate json report by default. Use --json-report to generate
the file.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-25 09:56:53 -05:00
Peter Bigot
affa7a1c7e Revert "device: add post-process of elf file to manage device handles"
This reverts commit 40d3653758.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2021-01-23 18:01:03 -05:00
Torsten Rasmussen
774103d050 cmake: pristine.cmake requires SOURCE_DIR and BINARY_DIR as arguments
Calling cmake/pristine.cmake now requires SOURCE_DIR and BINARY_DIR as
arguments.

This ensures that pristine.cmake can evaluate if pristine is requested
on in-source builds, and bail out in such case with an error message.

All uses of `pristine.cmake` has been updated to use the new arguments.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-01-23 08:31:13 -05:00
Daniel Leung
d3218ca515 debug: coredump: remove z_ prefix for stuff used outside subsys
This removes the z_ prefix those (functions, enums, etc.) that
are being used outside the coredump subsys. This aligns better
with the naming convention.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-01-21 22:08:59 -05:00
Daniel Leung
1387e7c7c2 scripts: coredump: fix crashing on non-existent memory region
get_mem_region() may return None for a non-existent memory region
so we need to check first before looking at its properties.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-01-21 22:08:59 -05:00
Kumar Gala
589775f14f requirements: Update to pyocd 0.29 for LPC55S69 support
Pyocd 0.29 supports being able to flash newer LPC55S69 as it handles
the debug unlock handshake.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2021-01-21 15:53:22 -05:00
Peter Bigot
40d3653758 device: add post-process of elf file to manage device handles
Following the idiom used for system calls, add script support to read
the initial application binary to identify which devices are defined,
and to use their offset in the device array as their unique handle
rather than the externally-defined ordinal from devicetree.  The
device dependency arrays are updated to use these handles.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2021-01-21 14:49:04 -06:00
Torsten Rasmussen
3d88083bf1 cmake: zephyr modules: sanitize all module name when used as variable
The introduction of Zephyr module glue code in the Zephyr repository
introduces a Kconfig variable in the form of:
`config ZEPHYR_<MODULE_NAME>_MODULE`.

All Kconfig variables go into `autoconf.h`, therefore it is necessary
to sanitize the Kconfig variable, so that it does not contain special
characters. To ensure consistent variable name, then the module name
will be sanitized in all variable use in both Kconfig and CMake.
The sanitization is done be replacing all special characters with an
underscore, `_`.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-01-20 12:59:19 +01:00
Torsten Rasmussen
1bd0b29918 cmake: scripts: supporting gen_relocate_app feature in windows
Fixes: #28847

This commit fixes two places that was causing gen_relocate_app to fail
in windows.

gen_relocate_app.py now splits only on first `:` in
`<MEM_REGION>:<file>`.

Windows contains `:` in path, thus only first `:` is valid for splitting
mem region and file path.

Second part of the issue is fixed in CMake where `'` was used for
quoting of command arguments.
This causes a file not found on Windows because the final `'` would be
treated as part of the filename. Similar the first `'` would be treated
as path of the mem region name.

This is fixed by using `"` for quoting, which works correctly on all
platforms.

gen_relocate_app.py:403: UserWarning: File: .../kernel/sem.c' Not found
Note the stray `'`                                         ^^^

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-01-20 11:51:47 +01:00
Torsten Rasmussen
3361b154ff scripts: check_compliance.py: include known Zephyr Kconfig glue code
Fixes: #31436

This fix adds ZEPHYR_<MODULE_NAME>_KCONFIG variable to compliance check.
This ensures that Zephyr Kconfig glue code is included in the
compliance check, and thus remove the issue reported in #31436.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-01-20 11:21:16 +01:00
Anas Nashif
989ebf6c35 kernel: add vrfy hooks to support userspace with condvar
Add needed vrfy hooks for userspace support.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-19 08:55:47 -05:00
Andrzej Głąbek
02819489a2 scripts: module: Add missing uses of the module name field
This is a follow-up to commit f24f88324c.

Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
2021-01-19 10:17:56 +01:00
Dominik Ermel
b493acedea scripts/requirements: Add required version to imgtool in extras
The minimal version of imgtool has been set at 1.7.1

Signed-off-by: Dominik Ermel <dominik.ermel@nordicsemi.no>
2021-01-18 16:59:29 -05:00
Marc Herbert
38eed65b2a scripts: move pip pykwalify from requirements-build-test to -base
... because zephyr_module needs it.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2021-01-18 16:56:24 -05:00
Torsten Rasmussen
f24f88324c scripts: module: support for name field in zephyr/module.yml
The folder name of a Zephyr module is also used as its module name
when integrating it into the build system.

This means that a Zephyr module, BAR, located in:
<workspace>/modules/foo
                    |--- zephyr
                          |--- CMakeLists.txt
                          |--- Kconfig

will be referred to as FOO in the build system, that is:
ZEPHYR_FOO_MODULE_DIR==<workspace>/modules/foo
ZEPHYR_FOO_CMAKE_DIR==<workspace>/modules/foo/zephyr

The `name` field allows the module to specify its module name,
independent of its location like:

<workspace>/modules/foo/zephyr/module.yml
```
name: bar
build:
  cmake: zephyr
```

will instead be referred to as BAR in the build system, that is:
ZEPHYR_BAR_MODULE_DIR==<workspace>/modules/foo
ZEPHYR_BAR_CMAKE_DIR==<workspace>/modules/foo/zephyr

This allows for greater flexibility of relocating Zephyr modules in
other folders and at the same time be guaranteed that other modules
depending on `ZEPHYR_BAR_MODULE_DIR` is still working.

If `name` field is not specified in `module.yml`, then the existing
behavior of using the folder name will be used.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-01-18 21:53:20 +01:00
Torsten Rasmussen
3673e288bd modules: introducing MODULE_EXT_ROOT to allow glue code in Zephyr repo
This commit introduces MODULE_EXT_ROOT which allows CMake and Kconfig
glue code to be placed outside of the Zephyr module repository.

This allows for placing glue code in Zephyr, but also allows users to
specify custom MODULE_EXT_ROOTs for glue code using either
`-DMODULE_EXT_ROOT` or `zephyr/module.yml` with
`build:settings:module_ext_root` settings.

MODULE_EXT_ROOT' is a list of directories, similar to other roots such
as BOARD_ROOT, DTS_ROOT, etc.
The Zephyr repo folder ${ZEPHYR_BASE} is always to the MODULE_EXT_ROOT
list as lowest priority.
For each MODULE_EXT_ROOT, the file
`<module_ext_root>/modules/modules.cmake` will be processed.

In Zephyr repo, the folder `modules/<module>/` contains CMakeLists.txt
and Kconfig glue code for the Zephyr module.

A Zephyr module can specify that CMakeLists.txt and Kconfig glue code is
placed in an external module root by specifying:
```
build:
  cmake-ext: True
  kconfig-ext: True
```

It is still possible to place the CMakeLists.txt and Kconfig files
directly in the Zephyr module using the existing:
```
build:
  cmake: <path>
  kconfig: <file>
```.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-01-18 18:02:46 +01:00
Anas Nashif
b1080ea036 twister: remove test type from json
This is always the same, so we might as well just remove it for now and
save some space since it is the default anyways. Type can be added later
when it provides more value.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-18 08:30:49 -05:00
Anas Nashif
cd1dccf9bc twister: fix json report
Fix JSON report generation and simplify it a bit.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-18 08:30:49 -05:00
Anas Nashif
211ef41125 twister: debug which coverage tool is being used
Add more debug messages about coverage tool being used.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-18 08:30:49 -05:00
Martí Bolívar
c19c6fb438 Revert "scripts: west build: default build.pristine to auto"
This reverts commit c505ca38cf.

Fixes: #31358

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-01-17 08:58:54 -06:00
Martí Bolívar
f7d33f2988 scripts: dts: gen_defines: don't include descriptions
Before we had a bindings index in the documentation, the generated
header file was (somewhat unfortunately) often our best reference for
what a particular binding or property within a binding ends up doing,
so it made good sense to put the description in the generated file.

Now that we have HTML documentation that's a bit more digestible than
the generated file, though, we can just point users at that. Do that
and remove the inline description from the generated file.

This makes it possible to put C-style multiline comments in the
descriptions themselves, which will be done in subsequent patches.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-01-16 07:59:27 -06:00
Martí Bolívar
00ffc7e7b7 devicetree: add DT_NODE_PATH()
This macro returns a node's full path, given its node identifier.

The entire path to a node is useful information for the user which can
be added to build-time error messages.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-01-16 07:59:27 -06:00
Martí Bolívar
0d5e6c13e9 boards/shields: re-work handling in cmake and west
Remove the boards and shields lists from the 'usage' target output.
That might have been readable at some point long ago in Zephyr's
history, when only a few boards were available, but right now it's
obscuring the high level targets we really want 'usage' to print.

Instead, add 'boards' and 'shields' targets which the user can run to
get those lists, and reference them from the 'usage' output. This
makes 'usage' squintable again. We use the new list_boards.py script
from the 'boards' target.

Reference the 'help' target from 'usage' as well, and drop the
recommendation that people run '--target help' from the 'west build
--help' output for the 'west build --target' option. The canonical
place to look is 'usage' now.

Use the new list_boards.py code from 'west boards' as well, which
allows us to add the board's directory as a format string key, in
addition to its name and architecture.

Keep west-completion.bash up to date. While doing that, I noticed that
a bunch of references to this file refer to a stale location, so fix
those too.

Finally, the 'usage' output is what we print for a failed board or
shield lookup, so that needs to be updated also. Handle that by
invoking boards.cmake and a new shields.cmake in CMake script mode to
print the relevant output.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-01-15 15:07:49 -05:00
Martí Bolívar
8bd6d08b0b scripts: add list_boards.py
This script is essentially a Python rewrite of the CMake code we're
using to print boards in cmake/boards.cmake, plus some extra features.

Having this in Python will simplify some later adjustments to our
'usage' build system target in ways that will make its output easier
to read, while simultaneously making 'west boards' more useful.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-01-15 15:07:49 -05:00
Erwan Gouriou
b711028539 scripts/Kconfig: Add dt_node_has_prop Kconfig function
Add dt_node_has_prop function to query the presence of 'prop'
for given node label.

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2021-01-15 10:29:30 -05:00
Kumar Gala
98b6e4f834 devicetree: generate extern's for devicetree struct devices
Generate a header (device_extern.h) that handles extern of possible
device structs that would come from devicetree.  This removes the need
for DEVICE_DT_DECLARE and DEVICE_DT_INST_DECLARE which we can remove.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2021-01-15 07:16:21 -06:00
Anas Nashif
5d50797dad ci: handle checpatch warnings as errors
Fail CI if we have both errors and warnings.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-14 17:29:37 -05:00
Anas Nashif
827ecb7bd0 twister: fix build_on_all configuration
Logic for scope of platforms to be checked was changed and this options
now only applies to limited scope, fix this and reset scope when a
testcase declares it needs to build on available platforms.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-14 13:59:42 -05:00
Kumar Gala
3befd7d45f cleanup: rename sanitycheck references to twister
Cleanup references to sanitycheck that should now be twister.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2021-01-14 07:50:52 -06:00
Martí Bolívar
7745f3edba west boards: add --name filter
This is an optional regular expression that can be used to filter the
output.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-01-14 12:39:21 +01:00
Marc Herbert
6d36386e13 requirements-compliance: restrict junitparser version < 2
junitparser version 2 is incompatible with check_compliance.py, it fails
like this:

File "./scripts/ci/check_compliance.py", line 295, in parse_kconfig
   self.skip("Not a Zephyr tree (ZEPHYR_BASE unset)")
File "./scripts/ci/check_compliance.py", line 141, in skip
   self.case.result = Skipped(msg, "skipped")
File "/usr/local/lib/python3.9/site-packages/junitparser/junitparser.py"
   line 682, in result
   for entry in value:

TypeError: 'Skipped' object is not iterable

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2021-01-13 07:58:37 -05:00
Marc Herbert
205dcdf636 check_compliance: fix broken format() in exception handler
Likely an accident when commit 288ae28c13 moved the script to the main
repo. Stayed unnoticed because this handler rarely ever runs.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2021-01-13 07:58:37 -05:00
Guennadi Liakhovetski
3de40b48c3 rimage: update rimage: add configuration and extended manifest
rimage dropped its "-m" parameter and switched over to using "-c"
for a configuration file, including a target name.

Add support for extended manifest for all cAVS versions.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
2021-01-11 16:10:23 -05:00
Marc Herbert
c6067e7f1c requirements-doc: add missing pkwalify
doc/CMakeLists.txt uses zephyr_module

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2021-01-11 14:51:45 -05:00
Hake Huang
d5c60ff2f1 test_version: update tset version with 12 digits
update the version with 12 digits in aligned with uniq request from

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
2021-01-10 17:00:17 -05:00
Eugeniy Paltsev
79b3a50772 twister: BinaryHandler: call try_kill_process_by_pid in a 'with' scope
The commit
531fe89e80 (sanitycheck: use multiprcoessing instead of threads)
introduce regression for ARC nsim_hs_smp platform.

The regression is that verification on nsim_hs_smp hangs. That
happens because now we don't call try_kill_process_by_pid()
in the 'with' scope of the 'subprocess.Popen' when we spawning
BinaryHandler thread. Previously it was called via `terminate`
method of `BinaryHandler` but it was changed in 531fe89e80.
So if we can't terminate the simulator in a normal way
(which is expected for 'mdb' which is used for nsim_hs_smp
simulation) we will hang forever - we will never return from
'with' scope of the 'subprocess.Popen' as we are waiting for
process termination but the try_kill_process_by_pid() is located
latter and we never reach it.

Fix that.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
2021-01-10 15:42:48 -05:00
Anas Nashif
b2a5782570 ci: remove west extension call
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-08 07:49:19 -05:00
Johan Hedberg
4ba387829b edtlib: Match any parent bus when binding lacks an explicit on-bus
There are some drivers in the tree that support devices on multiple
different buses, although so far this has not been represented in
device tree using the bus concept. In order to convert these drivers &
bindings to refer to a formal bus in device tree we need to be able to
match bindings which lack an explicit "on-bus: ..." value against any
parent bus. This will also be needed for any external bindings, since
those would not be aware of on-bus (as it's a Zephyhr-specific
extension).

The two drivers I'm particularly targeting is the ns16550 UART driver
(drivers/serial/uart_ns16550.c) and the DW I2C driver
(drivers/i2c/i2c_dw.c). They both support devices with a fixed MMIO
address as well as devices connected and discovered over PCIe. The
only issue is that instead of encoding the bus information the proper
DT way these bindings use a special "pcie" property in the DT node
entries to indicate whether the node is on the PCIe bus or not.

Being able to convert the above two drivers to use the DT bus concept
allow the removal of "hacks" like this:

 if DT_INST_PROP(0, pcie) || \
       DT_INST_PROP(1, pcie) || \
       DT_INST_PROP(2, pcie) || \
       DT_INST_PROP(3, pcie)

to the more intuitive:

 if DT_ANY_INST_ON_BUS_STATUS_OKAY(pcie)

This also has the benefit that the driver doesn't need to make any
arbitrary assumptions of how many matching devices there may be but
works for any number of matches. This is already a problem now since
e.g. the ns16550 driver assumes a maximum of 4 nodes, whereas
dts/x86/elkhart_lake.dtsi defines up to 9 different ns16550 nodes.

Signed-off-by: Johan Hedberg <johan.hedberg@intel.com>
2021-01-07 20:07:12 +02:00
Anas Nashif
f1dea627cc ci: fetch sof submodules
Fetch submodules needed to build with SOF.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-06 07:53:46 -06:00
Martí Bolívar
a8612eb308 runners: fix nrfjprog UICR check
It's fine to flash an image with UICR data as long as the user has
given any of the --erase, --recover, or --force options. Silence the
error correctly.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-01-05 08:44:05 -06:00
Jingru Wang
b9c953c126 twister: add 12 hexadecimal digits to version
With the earlier version of Git, git describe command will
use the default 7 hexadecimal digits as the abbreviated
object name. But currently the Linux kernel project needs
11 to 12 hexdigits to stay unique.So it's necessary to add
--abbrev=12 to ensure twister always generate the same tags.

In https://testing.zephyrproject.org/daily_tests/versions.json,
all tags have 10 hexadecimal digits.

Signed-off-by: Jingru Wang <jingru@synopsys.com>
2021-01-04 08:42:40 -05:00
Fabio Utzig
7cc84af393 doc: fix broken text search
Require Sphinx<3.4.0 to avoid the issue described in:

https://github.com/sphinx-doc/sphinx/issues/8603

This requirement can be relaxed once a new sphinx-rtd-theme is
released.

Signed-off-by: Fabio Utzig <fabio.utzig@nordicsemi.no>
2021-01-04 07:57:07 -05:00
Eugeniy Paltsev
3134bc1ea0 runners: nsim: allow to use runner if gdb is missing
As of today we check for gdb presence in 'do_create' method and
raise an exception in case of gdb is missing. That makes nsim
runner unusable without gdb even for the commands which don't use
it (like 'flash' command).

Fix that.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
2020-12-27 18:19:10 +01:00
Martí Bolívar
38ede5aded dts: handle unspecified phandle-array elements
The DTS language permits zeroing out phandles in a phandle array to
say "there's nothing at this index", and dtlib manages that correctly,
but edtlib and gen_defines.py aren't equipped to do so.

Fix this by allowing None elements in the lists of ControllerAndData
values returned by edtlib for such properties.

Handle that in gen_defines.py by setting the generated
DT_N_<node>_P_<prop>_IDX_<i>_EXISTS macro to 0 in such cases.

The DT_N_<node>_P_<prop>_LEN macro still accounts for the entire
length of the phandle-array; it's just that some indexes may be
missing data.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-22 07:34:29 -06:00
Martí Bolívar
42504e8c2c dts: tests: delete extraneous whitespace
This line is required by the commit linter.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-22 07:34:29 -06:00
Anas Nashif
4fa82c44af twister: fix hardware map load/save
Do not store shared memory variables into yaml and check for validity of
imported hardware map before attempting to modify it.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-19 19:20:09 -05:00
Anas Nashif
5be8afcf73 ci: update versions file with weekly label
Sync with version in test_results.
Set the version to be a weekly on Mondays.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-18 07:46:47 -05:00
Jingru Wang
f20c15caaf sanitycheck: Add exception check
Add exception check for psutil when running CI with nSIM

Sometimes nSIM random failures may turn nsimdrv process into zombie
process, parent process can't terminate it by calling proc.wait(0.1)
and kill command doesn't apply to zombie process

Signed-off-by: Jingru Wang <jingru@synopsys.com>
2020-12-17 09:18:00 -05:00
Jacob Trueb
278aca926b scripts: west_commands: runners: Extend jlink timeout
With Rosetta, jlink command runs longer than 0.1 seconds.

Signed-off-by: Jacob Trueb <jtrueb@northwestern.edu>
2020-12-16 14:26:13 +01:00
Anas Nashif
4594370b46 twister: remove some leftover sanitycheck mentions
Few leftover mentions of Sanitycheck.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-14 14:13:16 -05:00
Martí Bolívar
f673e2060c edtlib: warn on non-tokenizable enums
These are likely enough defined by mistake to emit a warning for.

Adjust tests to match, tweaking the test_warnings() setup: now that
we've got several test cases, it's a bit cleaner not to have to
copy/paste the ('edtlib', WARNING, ...) part of every expected log
record tuple.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-13 19:25:24 -05:00
Martí Bolívar
0985849aa2 edtlib: use standard logging module
The edtlib strategy for emitting warnings is to print directly to
standard error. This in turn requires hacks to drop stored references
to stderr in various _warn_file attributes so the EDT objects can be
pickled.

In general, I think it's not really appropriate for library modules
like edtlib to be printing to stderr directly. The user should be able
to configure logging for general utility data munging modules like
this as they please, and not just deciding what file to print to.

Move this around so the standard logging module is used instead. We
can preserve backwards compatibility in gen_defines by customizing the
'edtlib' logging module behavior so it prints the exact same thing it
always has.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-13 19:25:24 -05:00
Peter Bigot
345da78b5b devicetree: basic support for enum tokens
Whenever a devicetree binding defines a string property whose
enumerated values are all tokenizable, generate C macros for each
property value that are the corresponding tokens.

Note that "token" is distinct from "identifier": both 'foo' and '123'
are valid tokens, but only 'foo' is a valid identifier. We permit some
strings which are not valid identifiers in anticipation that the
generalization may be useful, e.g. when defining macros that paste the
token onto a prefix that makes the whole thing an identifier.

Fixes: #21273
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-13 19:25:24 -05:00
Martí Bolívar
b6dc0a24f7 edtlib: improve enum support
Add enum_tokenizable and enum_upper_tokenizable to PropertySpec. These
allow a PropertySpec to declare that it both has an enumeration of
values and all of them are strings which are "tokenizable". Don't
bother extending Property with these; the user can access the
information through Property.spec now, so the extra delegation is
unnecessary.

See the docstrings for details on what "tokenizable" means. The basic
idea is that we should be able to use the DT binding's enum values as
C 'enum' enumerators in a "reasonable way".

Add val_as_token to Property. This produces a canonical token for the
property value.

Add tests for this feature in particular and property enumerations in
general.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-13 19:25:24 -05:00
Martí Bolívar
88094ecfb1 edtlib: tweak Property initialization
In the 'normal' case of a property whose definition is taken from a
binding YAML file, a fair number (three at present) of the attributes
available on Property objects are directly taken from the
corresponding PropertySpec object.

Refactor the internals of how a Property gets initialized so that it
has a direct reference to its PropertySpec, and make those attributes
properties which just delegate to the PropertySpec (which in turn just
delegate to the binding). Additionally, expose the PropertySpec
directly.

This will make it easier to extend the Property class with additional
attributes that normally come from the PropertySpec without having to
touch all the locations where Property.__init__ is called.

In the case of the 'default' properties, we handle this by dummying
out some PropertySpec objects. These dummy PropertySpecs in turn
require a dummy Binding.

This change has the advantage that it improves the degree to which
these defaults are checked, e.g. it makes sure that 'status' is one of
the permitted values.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-13 19:25:24 -05:00
Anas Nashif
b517b1fc60 sanitycheck: add more logging
This will let us know what is going on, especially on operations that
take some time to complete.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 17:41:40 -05:00
Anas Nashif
2d4891725c sanitycheck: handle keyboard interrupts
Handle keyboard interrupts (Ctrl-C) and terminate gracefully.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 17:41:40 -05:00
Anas Nashif
a5d16ab027 sanitycheck: add option to store platform reports
Now we create plaform reports for each enabled platform, this is time
consuming, especially if you are not using those reports for anything.

Make this an option and enable only if the generated reports are going
to be used.

Individual reports can also be generated from the JSON file if needed.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 17:41:40 -05:00
Anas Nashif
7822e7b50d sanitycheck: retry-failed comment
Add a comment about how deal with retries when a test fails to build.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 17:41:40 -05:00
Martí Bolívar
8fce3004fe runners: mdb: fix imports
Runner implementations are only allowed to unconditionally import
modules in the python standard library. They are not allowed to import
anything that comes from pip or other third party sources unless they
catch the ImportError and gracefully do nothing.

Fix the imports in the mdb runner accordingly, sorting the imports
into sections to make it clearer what's stdlib, what's runners, and
what's third party while we're here.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-11 16:49:59 -05:00
Anas Nashif
487af47f58 twister: add a wrapper script with old name
Add a simpel wrapper script called 'sanitycheck' to be removed after
Zephyr 2.5.

This script has a warning and a 5 sec delay to get the attention of the
caller.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
5471398ff0 modules: rename options for twister support
Zephyr modules with test support should also be using twister instead of
sanitycheck.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
5725ed8dd1 twister: fixed pylint issues
Fixed import and unused variable warning.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
9318e6c8f7 twister: tests: rename sanitycheck -> twister
Change all tests to use twister instead of sanitycheck.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
a05d056943 ci: rename sanitycheck -> twister
Change all CI scripts to use the new script name: twister.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
2e4b0fad98 twister: adapt platform schema
Adapt schema to use twister instead of sanitycheck.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
92c9d4ba14 twister: move unit tests
Move all unittests to a new folder with the twister name.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
b18f311b3e twister: change all occurances of sanitycheck
mass search/replace for sanitycheck replacing it with the new name:
twiser.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
2d351e7ffc sanitycheck: rename main script
We are now known as twister.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
e508babfed sanitycheck: move sanity_chk -> pylib/twister
Move the "module" classes used by sanitycheck into a new location under
pylib, where we can have other python libraries reside instead of having
them all over the place.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
303fc90311 sanitycheck: remove obsolete .gitignore
Remove obsolete .gitignore file

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
94f6826d28 sanitycheck: move release data to release/
Move the CSV files with release related data out of the python directory
into scripts/releases, which is more appropriate.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
c24bf6f183 sanitycheck: move schemas to scripts/schemas
Move yaml schemas into one single location. We have quite a few schema
used across the tree, one common place will make them easier to find and
reference.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Martí Bolívar
6628a16e4d runners: nrfjprog: boilerplate and recover rework
Rework the runner to improve various issues.

Every board.cmake file for an nRF SoC target is repeating boilerplate
needed for the nrfjprog runner's --nrf-family argument. The
information we need to decide the --nrf-family is already available in
Kconfig, so just get it from there instead. Keep the --nrf-family
argument around for compatibility, though.

This cuts boilerplate burden for board maintainers.

We also need to revisit how this runner handles recovery to fix it
in nRF53 and keep things consistent everywhere else.

To cleanly handle additional readback protection features in nRF53,
add a --recover option that does an 'nrfjprog --recover' before
flashing. Keep the behavior consistent across SoCs by supporting it on
those too. Because this is expected to be a bit tricky for users to
understand, check if a --recover is needed if the 'nrfjprog --program'
fails because of protection, and tell the user how to fix it.

Finally, instead of performing a separate 'nrfjprog --eraseall', just
give --chiperase to 'nrfjprog --program' process's arguments instead
of --sectorerase. This is cleaner, resulting in fewer subprocesses and
avoiding an extra chip reset.

Having a separate 'west flash --recover' option doubles the number of
test cases if we want to keep exhaustively enumerating them. That
doesn't feel worthwhile, so update the test cases by picking a
representative subset of the possibilities. Each test now has enough
state that it's worth wrapping it up in a named tuple for readability.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-09 15:00:24 -06:00
Anas Nashif
371a9cc403 sanitycheck: fix json file generation
script was failing during a run with -f, the status key was not in the
dictionary and the check would cause an exception. Fix that using
.get().

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-09 08:21:02 -05:00
Gerson Fernando Budke
80d7253a47 scripts: runners: bossac: Fix offset parameter
The current bossac runner have multiple entries for flash offset
parameter.  Remove offset parameter from command line and rework
all infraestructure to get offset from device tree.  Add proper
verification routines to validate configurations on board and
device tree entries to fix SDK compatibility checks.

Fixes #29312.

Signed-off-by: Gerson Fernando Budke <nandojve@gmail.com>
2020-12-03 10:50:23 -06:00
Gerson Fernando Budke
94171c0ba2 scripts: runners: bossac: Extract make_bossac_cmd
Move bossac command construction outside do_run method to keep
clean code.

Signed-off-by: Gerson Fernando Budke <nandojve@gmail.com>
2020-12-03 10:50:23 -06:00
Gerson Fernando Budke
ffed132e82 scripts: runners: bossac: Change speed by protocol
Add is_extended_samba_protocol method to verify if board is using SAM-BA
extended protocol (Arduino/Adafruit UF2).  It allows enforce serial line
speed to 1200 baud automatically.  This avoid add entries on board.cmake
file.

Signed-off-by: Gerson Fernando Budke <nandojve@gmail.com>
2020-12-03 10:50:23 -06:00
Gerson Fernando Budke
876d8632a2 scripts: runners: bossac: Extract set_serial_config
Move serial settings outside do_run method to keep clean code.

Signed-off-by: Gerson Fernando Budke <nandojve@gmail.com>
2020-12-03 10:50:23 -06:00
Henrik Brix Andersen
c08d1e05fb sanitycheck: Add dt_chosen_enabled devicetree filter function
Add a dt_chosen_enabled(chosen) filter function to sanitycheck. The
function returns true if the devicetree /chosen node contains 'chosen'
and the referenced node is enabled.

Signed-off-by: Henrik Brix Andersen <henrik@brixandersen.dk>
2020-12-02 23:20:32 -05:00
Eugeniy Paltsev
9858893ea8 ARC: runner: mdb: tweak searching for cld process pid
mdb binary starts several subproceses and one of them is cld process.
In runners/mdb.py we record process id of cld on each mdb launch
to terminate simulator correctly later. However we can finish test
and terminate mdb before the cld process was found (so cld won't
be terminated correctly by sanitycheck infrastructure). It may happen
if we launch mdb on fast host machine.

That leads to several issues. First of all we get ugly error in
sanitycheck output:
------------------------>8--------------------------------
FileNotFoundError: [Errno 2] No such file or directory: '/xxxx/mdb.pid'
------------------------>8--------------------------------

Secondly (and it's more important) we terminate simulator incorrectly.
We terminate mdb leaving cld process alive, running and consuming one
cpu core permanently (until we kill it manually)

So, let's increase granularity of lookups and don't wait extra 0.5
seconds before the first lookup.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
2020-12-02 14:08:26 -05:00
Anas Nashif
f68146f74e sanitycheck: add option to treat overflows as errors
Add a new option that will enable treating RAM/SRAM overflows as errors
instead of skipping them, which is the default.

Fixes #27583

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-01 15:36:19 -05:00
Anas Nashif
8305d1b6ef sanitycheck: fix hardware map defaults
We were setting all devices as connected, now do that based on serial
availability.
Rename internal variables and make them shorter:

connected_hardware -> duts
ConnectedDevice = DUT

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-01 15:36:19 -05:00
Anas Nashif
a44268f4fb sanitycheck: tests: simplify filter testcase
use a loop instead of all() to make the test easier to debug.
remove the default platform test, as this is now being filtered
differently in the code.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-01 15:36:19 -05:00
Anas Nashif
ad70a69f85 sanitycheck: cleanup hardware map handling
Shorten class method names and print hardware map as before after
scanning.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-01 15:36:19 -05:00
Anas Nashif
d8e4a86788 sanitycheck: schema: 'available' is a runtime key
The available key is a runtime variable, it does not need to be in the
hardware map. Make it optional to keep the file format compatible for
now.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-01 15:36:19 -05:00
Anas Nashif
3b939da42c sanitycheck: unify terminology and fix accounting
Try and keep terminilogy consistent, a test application with multiple
test cases is now a 'test suite'.

Also fix accounting when retrying failed.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-01 15:36:19 -05:00
Anas Nashif
657b1ac354 sanitycheck: fix arch filtering
If we provide arch on command line, do not check for all platforms of
architectures not specified.

Fixes #30099

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-01 15:36:19 -05:00
Anas Nashif
f86bc058ce sanitycheck: reduce verbosity
Reduce debug messages while scanning directories for tests.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-01 15:36:19 -05:00
Anas Nashif
f909e99f04 sanitycheck: add some debugging during report phase
Show in debug mode what files are being written. In some cases this
takes a while and it is good to see what is going on.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-01 15:36:19 -05:00
Anas Nashif
696476e24e sanitycheck: report test name if no status
Also catch cases where we try to test on a device when no device is
connected.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-01 15:36:19 -05:00
Anas Nashif
bb42795644 sanitycheck: do not report about execution if --build-only
When running with --build-only, do not print messages about tests being
executed.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-01 15:36:19 -05:00
Anas Nashif
531fe89e80 sanitycheck: use multiprcoessing instead of threads
With python threading we have multiple issues with performance and
leakage. Use the python Process and implement locking using
multiprocessing library. This change improves performance and fixes
various issues with logging, concurrency and reliability of the output.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-01 15:36:19 -05:00
Anas Nashif
fe07d57e80 sanitycheck: error out if we do not find any tests
If we find no tests, error out instead of exiting silently.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-01 15:36:19 -05:00
Anas Nashif
743594faae sanitycheck: print pid of qemu process when debugging
For debugging, print the PID of the qemu running.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-01 15:36:19 -05:00
Torsten Rasmussen
ca08cc0d02 sanitycheck: check existence of testcase_extra.conf
Running sanitycheck for multiple platforms but only request coverage
on a single platform fails, example:
  sanitycheck --enable-coverage --coverage-platform nrf52840dk_nrf52840
  -p nrf52840dk_nrf52840 -p nrf52dk_nrf52832 -T <sample>

This happens because `testcase_extra.conf` will only be created if the
current platform is included in the list of coverage platforms.

The error in the example above that would be seen is:
  File not found:
      <...>/sanitycheck/testcase_extra.conf
This commit now uses the existence of `testcase_extra.conf` which is
created in the method `create_overlay()` before appending
the conf file to the list of overlay files.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-11-28 07:19:15 -05:00
Anas Nashif
82a6a467ec sanitycheck: do not try platforms not in hardware map
Get list of connected hardware and filter tests to only be run on
connected hardware in cases where no platform was specified when runn
the tests.  If the platform does not exist in the hardware map, just
skip it.

Fixes #29166

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-11-27 09:29:59 -05:00
Kumar Gala
52ad9a468c ci: run_ci.sh: retry failed west update for PR builds
We made a similar change for the nightly builds, but the PR builds
invoke run_ci.sh, so we need the same change here.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-11-23 14:42:54 -05:00
Spoorthy Priya Yerabolu
f8f220a9aa scripts: sanitycheck: Generate json report format
Generating JSON report format without changing
the existing report formats.
JSON report is divided into three sections
environment tests are run
test cases basic info and discarded tests
after applying filters
Added the complete debug log to the report

Fixes: #26443

Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
2020-11-20 11:44:29 -05:00
Jukka Rissanen
d1901d6a2f net: scripts: Make test runner script fully generic
Move actual test cases from the run-sample-tests.sh script to
the network samples directory that are supported by Docker based
testing. Each network sample directory that supports Docker testing,
will contain docker-test.sh script that is sourced by the runner
script. The docker-test.sh script will run the test as needed and
then return return value to the runner script.

Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
2020-11-20 16:07:59 +02:00
Martí Bolívar
4ac96079a7 west: flash/debug: try to make --context more prominent
I keep seeing people miss the --context option in the "west flash
--help" output.

This option is very important for understanding the runner-specific
options and state, and missing it means people get confused about what
west flash, debug, etc. can do and are doing.

Try to avoid this problem by adding a big fat banner about the
omission of runner-specific options in the main help output, and
provide more hints about how to use --context.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-11-17 16:38:36 -05:00
Anas Nashif
87e3ec4417 sanitycheck: test on integration platforms on top of default platforms
When running sanitycheck without any platform filters (-p option), we
take the default platforms and run on those only. When the -G options is
used for integration testing, the platforms listed as integration
platforms are not being added on top of default platforms.

This change adds integration platforms to the list an treats
integration_platforms as a positive filter, so that
integration_platforms are always being considered.

Fixes #29829

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-11-17 11:31:39 +01:00
Jukka Rissanen
66332cc648 net: scripts: Add dumb_http_server_mt sample to Docker testing
Add a way to run dumb_http_server_mt sample application against
HTTP get script (using curl) running in Docker.

Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
2020-11-17 12:25:36 +02:00
Jukka Rissanen
861c2c78f0 net: scripts: Add gPTP support to sample testing
Allow gPTP sample to be run via run-sample-tests.sh script.
This makes it possible to do simple smoke testing and verify that
gPTP stack is not broken.

Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
2020-11-17 12:15:37 +02:00
Eug Krashtan
a6be4f608e scripts: Add imgtool to requirements
On Windows, west sign command fails when imgtool not installed
via 'pip3 install'

Signed-off-by: Eug Krashtan <eug.krashtan@gmail.com>
2020-11-16 13:38:47 +01:00
Martí Bolívar
ffc03124c1 devicetree: make DT_PROP_HAS_IDX expand to 0 or 1
We have a use case for checking the results of a DT_PROP_HAS_IDX()
call with COND_CODE_1(). That won't work because its expansion is an
integer comparison; COND_CODE_1() expects a literal 1 or 0.

Adjust the macro implementation so it expands to a literal 1 or 0.
Make this work even when the index argument needs an expansion while
we're at it.

Fixes: #29833
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-11-15 08:44:56 -06:00
Martin Åberg
c107714214 sanitycheck: Add TSIM3 support
TSIM3 is a LEON processor and system simulator.

This commit allows using TSIM sanitycheck and the make run target.

TSIM parameters can be further specified in the board configuration
using TSIM and TSIM_SYS.

Signed-off-by: Martin Åberg <martin.aberg@gaisler.com>
2020-11-13 14:53:55 -08:00
Torsten Rasmussen
4558ba7128 cmake: ensure shields can be placed in other BOARD_ROOTs
Fixes: #28462

This commit allows shields to be defined in other BOARD_ROOTs, either
using `-DBOARD_ROOT=<path>` or a Zephyr module defined BOARD_ROOT.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-11-13 13:36:48 +01:00
Jukka Rissanen
d29ae958e9 net: scripts: Add http-client sample to Docker testing
Add a way to run http-client against http(s)-server.py that
is running in Docker.

Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
2020-11-13 07:25:27 -05:00
Eugeniy Paltsev
16032b67e8 sanitycheck: ARC: mdb-nsim: add missing stuff for simulation run
Add missing infrastructure for simulation run on mdb-nsim.
When the nSIM via mdb simulation support was added to Zephyr
scripts the handler call was lost. So nSIM via mdb simulation
was broken from the beginning.

Fix that bu adding missing handler call fro mdb-nsim.
Note that we use BinaryHandler which call west flash. It's
required in case of nSIM via mdb as we create mdb.pid file with
mdb-nsim runner help.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
2020-11-11 08:18:38 -05:00
Eugeniy Paltsev
9f2006c48b sanitycheck: ARC: nsim: allow to run tests on HW as well
ARC nSIM boards (starting with nsim_ prefix) allow to run
tests in simulator (nSIM). However with proper west runner
the tests can be run on real HW (FPGA based).

So check device_testing flag firstly and rollback to nSIM
simulation run in case of device_testing is not set.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
2020-11-11 08:18:38 -05:00
Eugeniy Paltsev
0699f38f16 ARC: west: mdb runner: record MDB's CLD process pid to file
Record MDB's CLD process pid to file so this process can be
terminated by sanitycheck infrastructure.
Update mdb runners test to be able to handle changes.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
2020-11-11 08:18:38 -05:00
Eugeniy Paltsev
91d7ec5a35 ARC: west: split mdb runner for mdb-hw & mdb-nsim runners
mdb runner is quite special as it can be used to run Zephyr on
both simulator (nSIM) and real hardware.
However it is really misleading as same command (west flash)
will run Zephyr in simulation for one board and try to run it
on HW for another board. Things are getting worse for boards
supporting both runs in simulation and on real hardware.

Let's split mdb runner for mdb-hw (for runs on HW) and mdb-nsim
(for runs in simulation) runners.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
2020-11-11 08:18:38 -05:00
Jukka Rissanen
2f4b66967d net: scripts: Close zephyr network interface when test closes
The zeth interface was left hanging around when the test
finished but we can remove it here as it is not needed.

Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
2020-11-09 14:42:34 +02:00
Jukka Rissanen
b68f7dfac2 net: scripts: Fix indentation and help usage
Fixing the indentation of run-sample-tests.sh script so that it
is easier to read.

Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
2020-11-09 14:42:34 +02:00
Gerson Fernando Budke
3fe4c702e7 runners: bossac: Add speed argument
The current stty command uses a hard code value of 1200.  This is not
compliant with SAM-BA specs and may create compatibility problems.  Add
an optional speed argument with 115200 as default value following SAM-BA
specifications.  All boards that needs a different speed should define
board_runner_args(bossac "--speed=<value>") with value as required
speed.

Signed-off-by: Gerson Fernando Budke <nandojve@gmail.com>
2020-11-06 12:15:55 -06:00
Gerard Marull-Paretas
e066a00d5e runners: stm32cubeprogrammer: add tests
Add tests for the stm32cubeprogrammer runner.

Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
2020-11-04 14:47:35 -06:00
Gerard Marull-Paretas
f98dd24993 runners: add support for stm32cubeprogrammer
Add support for the official ST Microelectronics programming tool (CLI
version).

Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
2020-11-04 14:47:35 -06:00
Gerson Fernando Budke
0d216924d5 samples: net: cloud: Introduce TagoIO IoT cloud http post
Introduce TagoIO IoT Cloud HTTP post client example.  This explorer
Zephyr network resources to demonstrate an end to end application.
The TagoIO allows that any user can test on a easy way Ethernet, WIFI
and Modem (PPP) with BSD sockets.  The example provides overlays to
configure WIFI and Modem.

The application consists an a pseudo temperature sensor that sends
periodically data to TagoIO IoT Cloud platform.  The data can be
visualized on a web browser dashboard, cellphone or tablet.  The
steps to configure TagoIO are described on the example documentation.

Special Variables:
 - CONFIG_TAGOIO_DEVICE_TOKEN   DEVID  token generated by TagoIO
 - CONFIG_TAGOIO_HTTP_WIFI_SSID SSID   when using WIFI
 - CONFIG_TAGOIO_HTTP_WIFI_PSK  PASSWD when using WIFI
 - CONFIG_MODEM_GSM_UART_NAME   UART   label when using MODEM
 - CONFIG_MODEM_GSM_APN         APN    when using MODEM

Signed-off-by: Gerson Fernando Budke <nandojve@gmail.com>
2020-11-04 09:06:23 +02:00
Martí Bolívar
0d4dca10b2 scripts: edtlib: child binding compatibles match parents
Fixes: #29758

Commit 7165b77a81 ("scripts: edtlib:
refactor for first class bindings") introduced a Binding class.

Its child_binding instance attribute has a compatible which can be
None. Adjust this behavior so the child Binding object has the same
compatible as the parent binding which ultimately has a compatible.

Without this, sanitycheck's expr_parser is doing some matching on
compatibles in child nodes that is producing unexpected results.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-11-03 14:58:51 -06:00
Martí Bolívar
2522eec578 scripts: testedtlib: add more child-binding tests
The child_binding object should default to having a path and
compatible that matches the parent's. Mark it as xfail because the
compatible part is failing.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-11-03 14:58:51 -06:00
Peter Bigot
abc72694ab doc: bump sphinx>=3.3.0
The filters were updated to match the 3.3.0 generated text, so update
the sphinx requirement to exclude versions that use different text.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2020-11-03 10:24:25 -06:00
Martí Bolívar
2c19cccd4b scripts: edtlib: bindings_from_paths() helper function
This is a convenience function for creating a bunch of Binding objects
from files in a directory.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-11-02 14:26:33 -05:00
Anas Nashif
be1025a736 sanitycheck: handle overflow skips
skips due to ram/rom overflow were captured as failures. Fix this and
count them correctly as skips.

Fixes #29412

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-11-02 14:21:55 -05:00
Anas Nashif
5a6e64f448 sanitycheck: count skipped tests due to overflow
Count tests that were skipped due to SRAM/RAM overdlow as skipped.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-11-02 14:21:55 -05:00
Anas Nashif
4cd63abafd ci: add more fields to versions.json
Add date and if a commit should be considered for weekly testing.
This will still work with the old format.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-10-30 15:56:52 +01:00
Kumar Gala
33db7b5b01 scripts: edtlib: Add support for 'deprecated'
Add the ability to mark a property as 'deprecated' to get a warning that
it will be removed in the future.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-10-29 11:06:32 -05:00
Fabio Utzig
8cd7ed1980 doc: bump breathe requirement to >=4.23.0
Breathe>=4.23.0 brings in basic xrefitem rendering support and fixes
issues parsing anonymous struct/unions. This version also adds a config
knob for showing or hiding enumerator values, so set it to generate
documentation compatible with previous versions.

Signed-off-by: Fabio Utzig <fabio.utzig@nordicsemi.no>
2020-10-27 13:09:24 +01:00
Maciej Perkowski
060e00d9c5 sanitycheck: Add checking of zephyr version
Adds function checking the version of zephyr, used for reporting

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2020-10-26 14:19:34 -04:00
Maciej Perkowski
725d19b79e sanitycheck: Add info about zephyr version to the xml reports
This commit adds posibility to save zephyr version in the xml reports.
It adds 'properties' attribute to testsuite attribute. 'properties'
can contain multiple 'property' attributes. Zephyr version is added
as such 'property'.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2020-10-26 14:19:34 -04:00
Andy Ross
60fb850713 scripts/west_commands: Don't demand non-empty output formats
It's not clear why this error is here.  The "formats" array seems to
be limited to "bin" and "hex" only, but every signing tool is going to
have its own idea of what format to emit and what ingredients need to
be used to do that.

In particular, rimage (used for the Intel Audio DSPs) doesn't use nor
generate zephyr.bin (it's very large), so it trips over this failure.

Just present the possibly-empty list of output formats to the Signer
object and let it make the decision about whether an empty formats
list is an error.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2020-10-23 22:22:52 -04:00
Anas Nashif
3ac163eae6 boards: rename up_squared_adsp intel_adsp_cavs15
The Audio DSP is not specific to up_squared, so make it more generic.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-10-23 12:56:03 +02:00
Erwan Gouriou
48b786b159 scripts: series-push-hook: Fix behavior after rebase
Avoids parsing the whole "before to now" commits after rebase.
Now, we just parse all commits since first commit of the branch
compared to master, each timer either at first push, forced push,
forced push after rebase.

Fixes #28509

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2020-10-21 07:16:54 -05:00
Kumar Gala
fc8c0c073c scripts: edt: Fix comment in _prop_val
_prop_val comment referred to one of the arguments called "optional"
however the code has changed to call that argument "required" now.  Fix
up the comment block to use the correct argument name and semantics of
that argument.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-10-21 06:44:32 -05:00
Kumar Gala
83be5cb9b3 scripts: edtlib: cleanup required arg to _check_prop_type_and_default
_check_prop_type_and_default doesn't use the required argument so remove
passing it to the function.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-10-21 06:44:32 -05:00
Kumar Gala
c7baf2f663 scripts: edtlib: Remove deprecated "category" support
"category" support has been deprecated for over 2 releases.  Removing
support for the keyword.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-10-21 06:44:32 -05:00
Martí Bolívar
876b961a71 scripts: edtlib: cosmetic variable name change
Change "elm" to "elem" for readability.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-10-21 06:44:32 -05:00
Martí Bolívar
7165b77a81 scripts: edtlib: refactor for first class bindings
Add two new types: Binding and PropertySpec.

- Binding is a first-class abstraction for a devicetree binding
  file as defined by a YAML file in the Zephyr syntax.

- PropertySpec is a helper type which represents a property
  definition within a Binding.

Make the Binding constructor a new entry point to the library. This
enables users to deal with bindings as standalone entities, apart from
how they characterize a particular devicetree.

Rework the EDT and Node internals that deal with bindings as dict
values to use the Binding type instead. To make this less ambiguous,
use the variable name 'raw' when we're dealing with a binding as it's
parsed from YAML, and 'binding' when we're dealing with a Python
Binding object.

This commit drops support for the following legacy bindings keys

- '#cells': use '*-cells' instead (e.g. 'gpio-cells', 'pwm-cells')
- "child-bus: foo" and "child: bus: foo": use "bus:" instead
- "parent-bus" and "parent: bus: ": use "on-bus:" instead

Officially, legacy bindings have been gone since
6bf761fc0a ("dts: Remove support for deprecated DTS binding
syntax"), so this is vestigial code, and I couldn't find any in-tree
users.

It also drops the convention in this file that ""-strings are
preferred.

I honestly don't understand why this was ever enforced; the file
itself admits single quotes are common in Python and we use them
elsewhere in Zephyr's Python code.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-10-21 06:44:32 -05:00
Martí Bolívar
8640a8d666 scripts: edtlib: tweak module docstring
Add a bit more information and clarification.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-10-21 06:44:32 -05:00
Anas Nashif
f751dd46b2 west: commands: sign: Add signing support for other CAVS targets
Sign for CAVS15, CAVS18, CAVS20 and CAVS25 SOCs

Originally-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-10-21 06:38:53 -04:00
Kumar Gala
eb2e89aecf sanitycheck: Fix bug with -M option
If we get a build failure with the -M option we get the following:

TypeError: 'NoneType' object is not iterable

This was due to having the default value of additional_keep in
cleanup_artifacts being None instead of an empty array.

Fixes #29376

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-10-20 19:30:43 -05:00
Kumar Gala
b2c07e439e sanitycheck: sanitize paths for device testing artifacts
To make the artifacts we keep around for --device-testing -M re-usable
we need to sanitize any aboslute build paths that might exist in
CMakeCache.txt or zephyr/runners.yaml.

This allows us to build the tests to run on one CI system and run the
tests on another system that has the board(s) connected to it.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-10-20 07:24:30 -05:00
Kumar Gala
1285c1f80a sanitycheck: make artifact cleanup for device-testing usable
If we specify --device-testing and -M make it so that the resulting
artifacts are usable to run with device-testing again.  This means
keeping around the zephyr binary images (zephyr.{bin, hex, elf}) and
a few files so 'west' can function for flashing (CMakeCache.txt and
zephyr/runners.yaml).

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-10-20 07:24:30 -05:00
Kumar Gala
037e1274d0 sanitycheck: qualify device testing output
If --device-testing and --build-only are specified surpress some board
related output

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-10-20 07:24:30 -05:00
Eugeniy Paltsev
ea11784919 sanitycheck: actually allow empty west-flash without parameter
According to west-flash option description it's allowed to use
west-flash without parameter:
> There are three ways this option is used.
> 1) bare: --west-flash
> 2) with a value: --west-flash="--board-id=42"
> 3) Multiple values: --west-flash="--board-id=42,--erase"

However, we don't allow to west-flash to be without parameter
when we verify sanitycheck arguments. Fix that.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
2020-10-16 06:26:21 -04:00
Torsten Rasmussen
fef7879583 scripts: ensuring posix path on module defined roots.
Fixes: #29235

This commit ensures that roots are being converted to posix paths.
This fixes the issue where windows style path (containing `\`) would
result in DTS dependency file to contain mixed style path separator and
thus causing Ninja to re-invoke CMake in an endless loop.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-10-15 22:07:11 -04:00
Martí Bolívar
305379e944 devicetree: add first round of dependency ordinal info
Add the first API functions that directly deal with node dependency
ordinals as determined by edtlib:

- DT_DEP_ORD(node_id): node_id's ordinal
- DT_REQUIRES_DEP_ORDS(node_id): list of dep ordinals for node_id's
  direct dependencies
- DT_SUPPORTS_DEP_ORDS(node_id): list of dep ordinals for nodes
  depending directly on node_id
- DT_INST_ equivalents

This is not meant to be an exhaustive set of macros related to
dependency ordinals; rather, it's a starting out point meant to enable
initial struct device dependency tracking work. We can add more if
needed.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-10-14 13:18:44 -05:00
Martí Bolívar
f752c5eeec west build: fix --pristine on early build system failure
The west build command has historically tried not to rm -rf
directories that don't look like zephyr build directories. The way it
does this is by checking for the presence of a CMake cache
with a Zephyr-specific variable (ZEPHYR_TOOLCHAIN_VARIANT) in it.

The problem with this approach is that if the build system fails
before this cache variable is set, the directory doesn't look like a
zephyr build directory, and therefore west build won't make it
pristine even with --pristine=always, even though build directories
resulting from failed runs like that are almost certainly
irrecoverably broken and need to be made pristine before anything will
work.

This leads to users having to rm -rf their directories manually, which
is not so nice.

To avoid this from happening, just check for ZEPHYR_BASE, which is
set early on in ZephyrConfig.cmake in 'modern' zephyr build systems.
Keep the ZEPHYR_TOOLCHAIN_VARIANT check in place for compatibility.

We could consider being less selective and just using shutil.rmtree()
whenever we have --pristine=always, but that would be a bigger
behavioral change than I'm comfortable doing without a good reason.

Fixes: #28876
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-10-14 07:22:32 -05:00
Kumar Gala
659b24bdcf sanitycheck: Fix results for --cmake-only
If we run with --cmake-only we get the following:

`ERROR   - Unknown status None`

Fix this by treating no status set as "passed" for the --cmake-only
case.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-10-13 13:13:21 +02:00
Martí Bolívar
8165008f44 dts: remove legacy macro support
The legacy macros were first deprecated in Zephyr v2.3. Now that
Zephyr v2.4 has been released, that makes two releases where these
macros have been deprecated, so it's OK to remove them.

This leaves support for legacy binding syntax in place. Removing that
is left to future work.

We need to update various pieces of documentation related to flash
partitions that never got updated when the new API was introduced.
Consolidate this information in the flash_map.h API reference page,
since that's really where users will run into it. This also gives us
the opportunity to improve this documentation.

Adjust a couple of kconfigfunctions.py and sanitycheck bits to use
non-legacy edtlib APIs.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-10-09 08:45:38 -05:00
Michael Hope
bf04c29c26 runners: bossac: handle --help on the current git version of BOSSA
BOSSA 1.9.1 and before an errror on `--help`, but the current git
version returns 0.  Handle both cases.

Signed-off-by: Michael Hope <mlhx@google.com>
2020-10-09 08:22:38 -05:00
Michael Hope
c0055ed392 runners: bossac: calculate the offset instead of absolute address
BOSSA takes the offset within flash memory, not the absolute address.
This doesn't matter on most boards as the flash starts at zero but
does matter on the Due as the flash starts at +512 KiB.

Signed-off-by: Michael Hope <mlhx@google.com>
2020-10-09 08:22:38 -05:00
Martí Bolívar
1ea7bf0db0 scripts: dts: fix handling for phandle-arrays with *-cells=0
Each controller node in a phandle-array may set the number of cells in
a specifier as any nonnegative integer. Currently, we don't allow
this in edtlib in the case where there are multiple controllers in a
phandle-array property all of which have 0 cells in the relevant
specifier, which is not correct. Fix this, add a regression test, and
improve the error message while we are here.

Fixes: #28709
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-10-06 12:05:37 -05:00