Commit Graph

3336 Commits

Author SHA1 Message Date
Andy Ross
2906d1aa51 soc/intel_adsp: Build bootloader with Zephyr
The presence of a separate build for the bootloader code has always
been a wart with this platform.  Sharing of code between the two has
required great care. We've had bugs with mismatched include paths,
macro definitions and compiler flags, etc...  And of course it's not
possible for one to see the other; in theory we'd like the ability to
call back into IMR code after startup, to use the space for temporary
storage, etc...

So let's finally do it.  This really isn't that complicated when you
see it in isolation:

+ Move the module manifest metadata into an "rimage_modules.c", and
  put them in their own NOLOAD section where we can grab them later
  with objcopy.

+ Make a new "imr" memory region in the main linker and just paste the
  bootloader linkage (which is now using its own specific sections) in
  there.

+ After zephyr.elf is built and cache-remapped, we can extract the imr
  sections and the appropriate manifest for the bootloader rimage
  module, and then do the converse by excluding them for the main
  image module.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-12-14 18:43:05 -06:00
Lauren Murphy
ddfb3a193c debug: coredump: dummy window registers, assign an to arn
GDB server dummies WINDOWBASE to 0 and WINDOWSTART to 1
for both arches and sends An as corresponding ARn for ESP32
to get around packet size limitation. Fixes backtrace and
other issues causing GDB client to crash.

Signed-off-by: Lauren Murphy <lauren.murphy@intel.com>
2021-12-14 07:40:55 -05:00
Lauren Murphy
80927cac87 debug: coredump: add xtensa coredump gdbserver
Adds support for Xtensa to coredump custom GDB server for
the sample_controller and ESP32 SOCs.

Signed-off-by: Lauren Murphy <lauren.murphy@intel.com>
2021-12-14 07:40:55 -05:00
Torsten Rasmussen
1a51993192 scripts: support propagating workspace status to zephyr revision
The PR #39382 raised a discussion on build reproducibility and knowledge
of west projects being out of sync with the west manifest.

Similar to how `git submodules` will report the working tree dirty if
any of the submodules HEAD points to a SHA different than the one
recorded in the super project.

Based on this discussion this commit extends the Zephyr and manifest
repo (when `west` is used) revisions to include the concatenated states
of the workspace.

The workspace states are:
>  dirty: false / true
>  extra: false / true
>  off:   false / true

the final revision will become: <SHA>-<state1>-<state2>-<state3>

For example:
> zephyr:
>  path: /.../zephyr
>  revision: <SHA>-dirty-extra-off

or
> zephyr:
>  path: /.../zephyr
>  revision: <SHA>-extra

The `BUILD_OUTPUT_META_STATE_PROPAGATE` Kconfig setting is introduced
to provide user control of this behavior.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-12-14 12:32:10 +01:00
Torsten Rasmussen
64ec6ee3a3 scripts: build meta file workspace status extension
The PR #39382 raised a discussion on build reproducibility and knowledge
of west projects being out of sync with the west manifest.

Similar to how `git submodules` will report the working tree dirty if
any of the submodules HEAD points to a SHA different than the one
recorded in the super project.

Based on this discussion this commit extends the Zephyr build meta file
with overall workspace status included in the meta file.

It adds the following meta info to the build meta file:
> workspace:
>  dirty: false / true
>  extra: false / true
>  off:   false / true

A project using west and having an extra Zephyr module loaded not
controlled using git and a west project at a SHA different than the
SHA referenced by the manifest can look like:
zephyr:
  path: /.../zephyr
  revision: 863600cd0e
modules:
- name: mcuboot
  path: /.../bootloader/mcuboot
  revision: c61538748ead773ea75a551a7beee299228bdcaf
- name: local_module
  path: /.../local_module
  revision: null
west:
  manifest: /.../zephyr/west.yml
  projects:
  - path: /.../zephyr
    revision: 863600cd0e
  - path: /.../bootloader/mcuboot
    revision: c61538748ead773ea75a551a7beee299228bdcaf-off
  - path: /.../tools/net-tools
    revision: f49bd1354616fae4093bf36e5eaee43c51a55127
workspace:
  dirty: false
  extra: true
  'off': true

And without west:
zephyr:
  path: /.../zephyr
  revision: 863600cd0e
modules:
- name: hal_nordic
  path: /.../modules/hal/nordic
  revision: a6e5299041f152da5ae0ab17b2e44e088bb96d6d
- name: local_module
  path: /.../local_module
  revision: null
workspace:
  dirty: false
  extra: true

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-12-14 12:32:10 +01:00
Erwan Gouriou
82b290ef48 scripts/pylib/twister: Add sn option to stm32cubeprogrgammer runner
In order to enable use of stm32cubeprogrammer runner with twister,
add "sn" tool specific option which allows to provide target serial
number and hence select the target to flash when multiple ones are
connected to the host.

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2021-12-13 17:20:17 -05:00
Hake Huang
2f3b8b01aa twister: remove redundant required for generate-hardware-map
remove a redundate required for --generate-hardware-map

this fixing: #41066

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
2021-12-13 08:46:15 -05:00
Anas Nashif
9cb2bc2c01 actions: clang: do not test all boards when --platform is given
If we are invoking testplan for only one board, do not bother with other
board related changes, just generate plan for the specified platform.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-12-09 16:23:22 +01:00
Jordan Yates
9c98d4f2b7 scripts: gen_defines: zephyr,linker-region unique
Add checks to ensure that `zephyr,linker-region` property values are
always globally unique.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2021-12-09 16:23:03 +01:00
Mark Holden
1a697ccf59 coredump: add support for RISC-V
This adds the necessary bits in arch code, and Python scripts
to enable coredump support for RISC-V

Signed-off-by: Mark Holden <mholden@fb.com>
2021-12-08 08:54:32 -05:00
Hake Huang
5c2bdfbbad twister: reorder ungroupped args
reorder ungroupped args in alpha-beta order

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
2021-12-08 08:25:45 -05:00
Hake Huang
fd914d7fc1 twister: reorder group args
reorder group args at parser function beginning

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
2021-12-08 08:25:45 -05:00
Hake Huang
dfe5514afc twister: refactor for pylint checking
refact to code sgement for better coding style

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
2021-12-08 08:25:45 -05:00
Hake Huang
9ab4e364d0 twister: argparse group refactor
add more groups in twister opts

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
2021-12-08 08:25:45 -05:00
Martí Bolívar
6c65b15ce6 edtlib: wrap DTError
Don't let a malformed devicetree escape as a DTError. Wrap it in an
EDTError instead, so callers can just rely on the edtlib APIs as is
generally expected.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-12-07 13:39:37 -05:00
Martí Bolívar
7738977af4 dtlib: handle GCC linemarkers
A GCC linemarker of the form:

  # 1 "filename" 2 3 4

or so is not currently being handled, because the current regular
expression assumes the "flags" values (the numbers after "filename")
are limited to a single value. Tweak the regular expression to allow
for up to 4 flags, which is what GCC documents it may emit:

https://gcc.gnu.org/onlinedocs/gcc-10.2.0/cpp/Preprocessor-Output.html

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-12-07 13:39:37 -05:00
Martí Bolívar
4e783a363a dtlib: add _Token __repr__
Convert numeric IDs to symbolic token ID names for ease of debugging.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-12-07 13:39:37 -05:00
Martí Bolívar
17ae4fe05a scripts: gen_dts_cmake: emit properties for /aliases
This enables cmake extensions that can look up the path for any
devicetree alias, or check if the alias is missing, etc.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-12-06 08:01:48 -05:00
Martí Bolívar
b65b49bef6 scripts: gen_dts_cmake: a small optimization
Using edt.chosen_nodes looks like a simple attribute lookup, but it's
actually calling a property that will create a new list of chosen
nodes every time. Apply a small optimization by only creating the list
once.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-12-06 08:01:48 -05:00
Alex Kolosov
f374d6b1f1 scripts: runners: Add west flash command for B91 platform
This commit implements west flash command for Telink B91 platform.
west flash command uses ICEman and SPI burn from AndeSight for flashing.

Signed-off-by: Alex Kolosov <rikorsev@gmail.com>
2021-12-06 07:28:38 -05:00
Anas Nashif
77a7df31e5 ci: fix test plan for smaller sets
We were missing smaller sets and skipping tests when the set is smaller
than the allowed number of tests per node.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-12-05 10:53:50 -05:00
Tom Burdick
e29d9e1e2f checkpatch: Add UTIL_CAT support to checkpatch
Checkpatch was checking for macro concats (##), however
we often use UTIL_CAT to allow for multiple macro expansion passes
to succeed.

Signed-off-by: Tom Burdick <thomas.burdick@intel.com>
2021-12-04 07:22:56 -05:00
Trond Einar Snekvik
6a8326ff1e modules: Use posix paths in Kconfig module prompts
kconfiglib doesn't render backslashes in menu strings, and the prompts
in the module menu will be unintelligible on Windows. Print these
menus as posix paths on Windows as well, so they don't look broken in
menuconfig.

Signed-off-by: Trond Einar Snekvik <Trond.Einar.Snekvik@nordicsemi.no>
2021-12-03 16:50:31 -06:00
Gerard Marull-Paretas
92b9aff9b3 scripts: checkpatch: typedefsfile: add pinctrl_soc_pin_t
The pinctrl_soc_pin_t is used as an opaque type in pinctrl to store pin
configurations which are SoC dependent.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2021-12-03 13:21:40 +01:00
Anas Nashif
8d40928b2a actions: twister: determine nodes in python script
Improve calculation of matrix and move calculations from workflow to the
testplan script. We now generate a file that can be parsed by the action
with the data needed to start twister with the right number of nodes.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-11-29 20:06:24 -05:00
Anas Nashif
d52212b09a actions: fix filtering for clang action
Do not invoke --integration when dealing with one platform only and
generate testplan only for the needed platforms.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-11-29 20:06:24 -05:00
Yasushi SHOJI
b652fe2a7b scripts: west_commands: runners: openocd: Add --verify-only
Add yet another option, --verify-only, to west flash.  This option
runs openocd with verify_image but without load_image.

Signed-off-by: Yasushi SHOJI <yashi@spacecubics.com>
2021-11-29 09:35:57 -05:00
Yasushi SHOJI
da8a41d42a scripts: west_commands: runners: openocd: Add flash --verify
OpenOCD has verify_image command.  Add `--verify` option to west flash
so that openocd can do vierfy_image right after load_image.

Signed-off-by: Yasushi SHOJI <yashi@spacecubics.com>
2021-11-29 09:35:57 -05:00
Anas Nashif
8b55275d48 ci: test_plan: fix pylint warnings
Fix some of the warnings reported by pylint.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-11-28 20:35:40 -05:00
Anas Nashif
053cc9fe62 actions: twister: run tests on all platforms when changed
When tests are changes, run them on all supported platformed, not only
integration platforms.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-11-28 20:35:40 -05:00
Gerard Marull-Paretas
d47700177a scripts: utils: add pinctrl migration script for nRF boards
This script can be used to automatically migrate the Devicetree files of
nRF-based boards using the old <signal>-pin properties to select
peripheral pins. The script will parse a board Devicetree file and will
first adjust that file by removing old pin-related properties replacing
them with pinctrl states.  A board-pinctrl.dtsi file will be generated
containing the configuration for all pinctrl states. Note that script
will also work on files that have been partially ported.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2021-11-26 14:20:51 +01:00
Torsten Rasmussen
41c25a2de4 twister: remove CMAKE_EXPORT_COMPILE_COMMANDS=1
The -DCMAKE_EXPORT_COMPILE_COMMANDS=1 is removed from twisterlib.py as
it is now always set by the Zephyr build system.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-11-25 10:41:28 -05:00
Torsten Rasmussen
f643b8b369 scripts: support compile_commands.json in gen_app_partitions.py
Fixes: #40590

This commit updates gen_app_partitions.py to include only files present
in the current build by extracting the information from the CMake
generated `compile_commands.json` file.

This ensures that object files in sub-projects, such as `empty_cpu0`,
will not be considered by the script.

Using the compile_commands.json instead of walking the whole build tree
for finding object files also improves performance:

Time of executing `gen_app_partitions.py` (Old):
__________________________
Executed in  480.06 millis
   usr time  425.83 millis
   sys time   49.55 millis

Time of executing `gen_app_partitions.py` (New):
________________________________________________________
Executed in   76.22 millis
   usr time   49.00 millis
   sys time   24.59 millis

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-11-25 10:41:28 -05:00
Anas Nashif
0b2df8f41c actions: twister/clang: cleanup test plan generation
Merged 3 files used to generate the test plan per PR based on the
changes. 2 python scripts and a shell script are now all merged into 1
python script that generates the input file for twister based on a list
of changed files by the PR.

This remove lots of old and obsolete code and simplifies things a bit,
no need anymore for an intermediate script to call twister, we call it
directly in the workflow and use the new test_plan script to generate
the test plan.

This also reenables the recently disabled tag based filtering which had
a bug, bug is resolved in this new implementation.

On push events, we now run twister without the --integration option to
catch any issues in the main branch that were not caught in PRs. PRs
continue to run with --integration enabled. This event (push) is now run
on 15 builders due to the increased size.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-11-24 22:54:25 -05:00
Johann Fischer
73199f2c91 scripts/Kconfig: add dt_node_str_prop_equals Kconfig function
Add function to check if a property of a node
is equal to a passed string.

The motivation is that in current USB device support
the sizes of bulk endpoints are configured in Kconfig.
The information if a USB device controller supports
high-speed comes from devicetree. This information must be
mapped in controller driver Kconfig and corresponding options
in USB device stack configured must be adjusted.

Signed-off-by: Johann Fischer <johann.fischer@nordicsemi.no>
2021-11-24 12:28:02 +01:00
Anas Nashif
5c24fcb4ae actions: run twister using github action
This action replaces current buildkite workflow and uses github actions
to build and run tests in the zephyr tree using twister. The main
differences to current builtkite workflow:

- the action handles all 3 events: pull requests, push and schedule

- the action determines size of matrix (number of build hosts) based on
  the change with a minimum of 1 builder. If more tests are built/run
  due to changes to boards or tests/samples, the matrix size is
  increased. This will avoid timeouts when running over capacity due to
  board/test changes.

- We use ccache and store cache files on amazon S3 for more flexibility

- Results are collected per build host and merged in the final step and
  failures are posted into github action check runs.

- It runs on more powerful instances that can handle more load.
  Currently we have 10 build hosts per run (that can increase depending
  on number of tests run) and can deliver results within 1 hour.

- the action can deal with non code changes and will not allocate more
  than required to deal with changes to documentation and other files
  that do not require running twister

The goal long-term is better integrate this workflow with other actions
and not run unncessarily if other workflows have failed, for example, if
commit message is bogus, we should stop at that check, to avoid wasting
resources given that the commit message will have to be fixed anyways
which would later trigger another run on the same code.

Currently there is 1 open issue with this action related to a github
workflow bug where the final results are not posted to the same workflow
and might appear under other workflows. Github is working on this bug.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-11-23 06:42:23 -05:00
Anas Nashif
2c5a37cb74 scripts: gen_app_partitions: do not load empty files
Do not load empty files through the ELF parser and raise exception when
magic number of ELF is not matched.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-11-23 06:42:23 -05:00
Jordan Yates
106f710c7d device: iterable supported devices
Add supported device information to the device `handles` array. This
enables API's to iterate over supported devices for power management
purposes.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2021-11-23 12:17:14 +01:00
Fabio Baltieri
d0377d30e3 scripts: runners: add connect_srst support for blackmagicprobe
Black Magic Probe supports a "monitor connect_srst" command [1] to
configure whether to assert reset when connecting to the target. This is
useful to reprogram a target that may be idling in a low power state,
with an otherwise unresponsive debug core.

Adding a --connect-srst runner option for enabling this for "flash" and
"debug" operations to ensure that programming works in that case, but
also explicitly disabling it for "attach", to allow attaching to a
running target without changing its state.

Not turning this on by default since apparently some MCUs disable the
SWD interface while in reset.

[1] https://github.com/blacksphere/blackmagic/wiki/Useful-GDB-commands

Signed-off-by: Fabio Baltieri <fabio.baltieri@gmail.com>
2021-11-22 22:20:02 -05:00
Olof Johansson
16003b1e58 twister: Fix trivial typo
Plarforms -> Platforms

Signed-off-by: Olof Johansson <olof@lixom.net>
2021-11-22 08:32:12 -05:00
Julien Massot
c727a9a4ca scripts: gen_kobject_placeholders: iterating the dictionary directly
Fix:
************* Module gen_kobject_placeholders
scripts/gen_kobject_placeholders.py:88:19: C0201:
Consider iterating the dictionary directly
instead of calling .keys() (consider-iterating-dictionary)

Signed-off-by: Julien Massot <julien.massot@iot.bzh>
2021-11-19 19:16:11 -05:00
Julien Massot
36f116b47f scripts/arch: remove usage of deprecated LooseVersion
replace with version.parse from packaging module.

prevent this warning message:
DeprecationWarning: The distutils package is deprecated
and slated for removal in Python 3.12. Use setuptools or
check PEP 632 for potential alternatives

Signed-off-by: Julien Massot <julien.massot@iot.bzh>
2021-11-19 19:16:11 -05:00
Shao Ming
4315140702 twister: fix the testcase-schema.yaml
Change the "yes/no" value to "true/false".
Otherwise pykwalify 1.8.0 will report below errors:
- Value: 'no' for required keyword must be a boolean:
  Path: '/mapping/common'>
- Value: 'yes' for required keyword must be a boolean:
  Path: '/mapping/sample/mapping/name'>

Signed-off-by: Shao Ming <ming.shao@intel.com>
2021-11-19 11:46:38 -05:00
David Leach
1e5a830233 scripts: kconfig: Add unit divisor to dt_node_int_prop_int/hex functions
Added a unit divisor option to the dt_node_int_prop_int and
dt_node_int_prop_hex functions to allow retrieval of DTS items as
a size value of K, M, or G divisor similar to the dt_node_reg_size_int
function.

Signed-off-by: David Leach <david.leach@nxp.com>
2021-11-18 14:29:53 +01:00
Maureen Helm
525fa76f4d boards: xtensa: Use a CMake variable to set the rimage target name
Removes hardcoded logic in the west signing script that translates
Zephyr board names to rimage target names. Instead, use a cached CMake
variable set at the board level to define its respective rimage target
name. This eliminates the need to modify the west signing script when
new SOF-supported boards are introduced to Zephyr.

Signed-off-by: Maureen Helm <maureen.helm@intel.com>
2021-11-17 19:44:35 -05:00
Jordan Yates
44b8a0d199 scripts: gen_handles.py: remove size restrictions
With `gen_handles.py` now running on the first pre-built image,
`zephyr_pre0.elf` there is no requirement for the device handle arrays
to remain the same size after processing.

Remove the padding generated in `gen_handles.py`, as well as the
temporary option `CONFIG_DEVICE_HANDLE_PADDING` which was added to work
around this issue.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2021-11-16 10:41:59 +01:00
Anas Nashif
a5c2a6169b actions: footprint: fix csv formatting
Fix formatting of plan.txt CSV file and make it parseable by script.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-11-15 11:12:05 -05:00
Torsten Rasmussen
fffaf05e5d scripts: support for build meta file creation
This commit is motivated by the west discussion in:
https://github.com/zephyrproject-rtos/west/issues/548

The commit provides the ability to generate a build meta info file
containing lists of:
- Zephyr:         path and revision
- Zephyr modules: name, path, and revision
- West:           manifest path
                  path and revision for each project

For Zephyr or Zephyr modules the revision will be `null` if it is not
under git version control.

If Zephyr, a modules, or a project has uncommitted changes, the revision
will be marked dirty.

If west is not installed or used for the build process, the
west-projects list will be empty.

If a project is both a Zephyr module and a west project it will show up
in both lists.

Similar to Zephyr, which is independently referred as the Zephyr in use
but also listed as west project when west is used.
This is important in case ZEPHYR_BASE was manually set and pointing to
a different Zephyr repository.

The build meta file is not created per default but can be enabled with
the BUILD_OUTPUT_META Kconfig setting.

A project using west and having an extra Zephyr module loaded not
controlled using git can look like:
zephyr:
  path: /.../zephyr
  revision: 863600cd0e3c0a271e86629c5089821e5e4380cc-dirty
modules:
- name: mcuboot
  path: /.../bootloader/mcuboot
  revision: c61538748ead773ea75a551a7beee299228bdcaf
- name: local_module
  path: /.../local_module
  revision: null
west:
  manifest: /.../zephyr/west.yml
  projects:
  - path: /.../zephyr
    revision: 863600cd0e3c0a271e86629c5089821e5e4380cc-dirty
  - path: /.../bootloader/mcuboot
    revision: c61538748ead773ea75a551a7beee299228bdcaf
  - path: /.../tools/net-tools
    revision: f49bd1354616fae4093bf36e5eaee43c51a55127

And without west:
zephyr:
  path: /.../zephyr
  revision: 863600cd0e3c0a271e86629c5089821e5e4380cc-dirty
modules:
- name: hal_nordic
  path: /.../modules/hal/nordic
  revision: a6e5299041f152da5ae0ab17b2e44e088bb96d6d
west: null

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-11-11 06:06:07 -05:00
Anas Nashif
46e3318dbd ci: hotfix: disable test exclusion by tags
Many tests and CI activties are being missed by excluding tests
mistakingly when running twister.
This is visibile when you change one or more tests in kernel/ for
example, twister does not run those tests that have changed at all and
marking the PR as tested and ready to be merged.

Temporary fix for #40235.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-11-10 22:37:21 -05:00
Anas Nashif
23ab949975 ci: inclusive language, use nodes instead of slaves
Fix the usage of the word slaves and replace with nodes.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-11-10 08:04:26 -06:00
Marc Herbert
c2524a07a6 west: build.py: resolve symlinks in cached source directory check
Fixes the following error:

ERROR: Build directory "<whatever>" is for application "<symbolic source
  dir in CMakeCache.txt>", but source directory "<resolved source dir>"
  was specified; please clean it, use --pristine, or use --build-dir to
  set another build directory
FATAL ERROR: refusing to proceed without --force due to above error

Fixes initial commit ab82264ace ("scripts: add west build, flash, and
debug commands")

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2021-11-10 08:27:16 -05:00
Neil Armstrong
1e8f0f3bd1 gen_defines: generate ranges defines for PCIe I/O and memory regions
As described in IEEE Std 1275-1994, the PCIe bindings uses the ranges
property to describe the PCI I/O and memory regions.

Write _RANGES_ defines that will be used to determines the I/O and
memory regions from PCIe Controller drivers.

Also exclude "ranges" & "dma-ranges" property's length generation
alogn "reg" and "interrupt".

Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
2021-11-10 08:23:00 -05:00
Neil Armstrong
2f4a6646cf edtlib: add tests to check Range property
This adds some tests in test_edtlib.py and test.dts to check all
common possible combination of ranges property usage and handling
by edtlib.

Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
2021-11-10 08:23:00 -05:00
Neil Armstrong
cd6a5648d7 edtlib: add Range property on Node, used for PCIe I/O and memory regions
As described in IEEE Std 1275-1994, the PCIe bindings uses the ranges
property to describe the PCI I/O and memory regions.

Add parsing of this property in edtlib.

Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
2021-11-10 08:23:00 -05:00
Carles Cufi
5ba301d449 footprint: Add Bluetooth samples to footprint tracking
Add a few Bluetooth samples to track their footprint as part of the
effort to contain firmware bloat.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2021-11-10 08:05:21 -05:00
Gerson Fernando Budke
0de934b2a7 scripts: runner: Introduce gd32isp flash runner
Add GigaDevice ISP console flash runner.  This tool enable uses ROM
bootloader to flash devices using serial port.

The GD32_ISP_Console tool can be found at
  http://www.gd32mcu.com/download/down/document_id/175/path_type/1

Signed-off-by: Gerson Fernando Budke <gerson.budke@atl-electronics.com>
2021-11-07 05:20:50 -05:00
Piotr Golyzniak
8b773481e9 twister: reenable build samples on Windows OS
Fix few limitation which do not allow to run Twister on Windows OS.
Especially turn off "import pty" module, due to the fact, that this
module can be used only on POSIX OS.

Fixes #40007

Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
2021-11-04 07:33:39 -04:00
Bartosz Bilas
d85465fb11 scripts: gen_relocate_app: check if argument matches
Check each line whether contains the correct pattern
${location}:${file} before splitting because it
turns out that script passes also compile
definitions added in project CMakeLists.txt such as:

add_compile_definitions(ABCD="XYZ")

which results in the build failure.

Signed-off-by: Bartosz Bilas <b.bilas@grinn-global.com>
2021-11-02 22:26:41 -04:00
Bartosz Bilas
50ec0461af scripts: gen_relocate_app: fix C0201 pylint warning
As pylint says "Consider iterating the dictionary directly
instead of calling .keys()" don't use .keys() method.

Signed-off-by: Bartosz Bilas <bartosz.bilas@hotmail.com>
2021-11-02 13:26:09 +01:00
Anas Nashif
910e86af68 ci: do not run twister on footprint script changes
No need to run twister on changes to changes to scripts/footprint/*

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-10-29 12:26:36 -04:00
Anas Nashif
82ebf78f95 footprint: add power management application
Add the footprint application with PM enabled for tracking.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-10-29 12:26:36 -04:00
Yuval Peress
27f6a5e07d ztest: add a weak implementation of test_main()
Introduce a weak implementation of test_main() which calls:
* ztest_run_registered_test_suites(NULL);
* ztest_verify_all_registered_test_suites_ran();

This will attempt to run all registered test suites and verify that
they each ran.

Signed-off-by: Yuval Peress <peress@chromium.org>
2021-10-28 16:57:51 -04:00
Yuval Peress
dee79d2b66 ztest: Add register functionality
Add new functionality to ztest to improve test modularity. The two
primary new entry points are:
* ztest_register_test_suite
* ztest_run_registered_test_suites

When registering a new test suite, users provide the name as well as
an optional predicate used to filter the tests for each run. Using NULL
as the predicate ensures that the test is run exactly once (after which
it is automatically filtered from future runs).

Calls to ztest_run_registered_test_suites take a state pointer as an
argument. This allows the the pragma functions to decide whether the
test should be run.

The biggest benefit of this system (other than the ability to filter
tests and maintain a larger test state) is the ability to better
modularize the test source code. Instead of all the various tests
having to coordinate and the main function having to know which tests
to run, each source file manages registering its own test
suite and handling the conditions for running the suite.

Signed-off-by: Yuval Peress <peress@chromium.org>
2021-10-28 16:57:51 -04:00
Bartosz Bilas
1502c6631d runners: jlink: exit with error in case of a failure
Add "ExitOnError 1" argument that treats any command-error
as fatal thus in the case of a programming error the "west flash"
command will return the correct error code instead of the default 0. It
fixes the false positive return codes when e.g we call west flash
command without a connected programmer or with the disconnected board.

Signed-off-by: Bartosz Bilas <b.bilas@grinn-global.com>
2021-10-28 10:57:28 +02:00
Torbjörn Leksell
95ad27fe1f Tracing: TRACING_SYSCALL status ignored fix
Added check to respect the TRACING_SYSCALL tracing option
when generating syscall tracing overrides in
gen_syscalls.py.

Signed-off-by: Torbjörn Leksell <torbjorn.leksell@percepio.com>
2021-10-26 15:13:08 -04:00
Jim Shu
46eb3e5fce gen_app_partitions: add .sdata/.sbss section into app_smem
Some architectures (e.g. RISC-V) has .sdata/.sbss section for small
data/bss. Memory partition should also manage the permission of these
sections in library so they should be put into app_smem.
(For example, newlib _impure_ptr is in .sdata section and
__malloc_top_pad is in .sbss section in RISC-V.)

Signed-off-by: Jim Shu <cwshu@andestech.com>
2021-10-25 19:16:37 -04:00
Tom Burdick
97dc88bb6d tracing: Automatic syscall tracing
When generating syscall wrappers, call a tracing macro with the id,
name, and all parameters of the syscall as params when entering and
leaving the syscall. This can be disabled in certain call sites
by defining DISABLE_SYSCALL_TRACING which is useful for certain
tracing implementations which require syscalls themselves to work.

Notably some syscalls *cannot* be automatically traced this way and
headers where exclusions are set are in the gen_syscall.py as notracing.

Includes a systemview and test format implementation.

Tested with systemview, usb, and uart backends with the string
formatter using the tracing sample app.

Debugging the trace wrapper can be aided by setting the TRACE_DIAGNOSTIC
env var and rebuilding from scratch, a warning is issued for every
instance a syscall is traced.

Automatically generating a name mapping for SYSVIEW_Zephyr.txt is a
future item as is documenting how to capture and use the tracing data
generated.

Signed-off-by: Tom Burdick <thomas.burdick@intel.com>
2021-10-23 20:45:17 -04:00
Torsten Rasmussen
1c911290e9 kconfig: add support for warnings when using experimental features
This adds two new Kconfig settings.

The first setting `EXPERIMENTAL` which is a promptless symbol.
This symbol must be selected by any setting which is used to enable
an experimental feature.

The second setting is `WARN_EXPERIMENTAL` which is a user controlled
setting that configures the build system to print a warning when
experimental features are enabled.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-10-22 13:59:21 +02:00
Maciej Perkowski
2253c6efd8 twister: Fix skipping unskippable tests when --subset is used
Fixes a bug causing tests with error status get silently skipped
when --subset is used.

Fixes: #39619

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-10-21 21:45:53 -04:00
Iuliana Prodan
602e675418 west: sign: add support for NXP i.MX8MP
Add support for signing i.MX8MP SOF with
Zephyr images with rimage.

Signed-off-by: Iuliana Prodan <iuliana.prodan@nxp.com>
2021-10-20 19:08:50 -04:00
Henrik Brix Andersen
5d34a45697 runners: openocd: always search the board support directory if found
Always add the boards/<arch>/<board>/support directory to the OpenOCD
runner search path if the directory exists.

This simplifies using custom --config <partial-board.cfg> runner
arguments without having to use the full path to the cfg file.

Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
2021-10-19 17:38:54 +02:00
Steve Winslow
878ee1493e west: spdx: Fix --init for Windows builds
Currently, west spdx --init uses os.mknod to create an empty file
to enable the Cmake file-based API system. As reported in #39311,
Python on Windows does not implement os.mknod.

This commit switches to using open()/close() instead of os.mknod()
to address this issue.

Signed-off-by: Steve Winslow <steve@swinslow.net>
2021-10-18 09:47:30 -04:00
Christopher Friedt
8b55eb8052 scripts: get_maintainer: add count function
For looking at statistics over time, it is helpful to be able to
quickly query the number of unique maintainers, collaborators,
areas, and the number of orphaned areas (without a maintainer).

This change adds the `count` subcommand to
`scripts/get_maintainer.py`.

```
% python3 ./scripts/get_maintainer.py count -h
usage: get_maintainer.py count [-h] [-a] [-c] [-n] [-o]

optional arguments:
  -h, --help            show this help message and exit
  -a, --count-areas     Count the number of areas
  -c, --count-collaborators
                        Count the number of unique collaborators
  -n, --count-maintainers
                        Count the number of unique maintainers
  -o, --count-orphaned  Count the number of orphaned areas

% python3 ./scripts/get_maintainer.py count
areas:          113
maintainers:    49
collaborators:  81
orphaned:       21

% python3 ./scripts/get_maintainer.py count -o
orphaned:       21
```

Signed-off-by: Christopher Friedt <chrisfriedt@gmail.com>
2021-10-18 08:36:21 -04:00
Julien Massot
a35e046fb2 runners: openocd: prefer extended-remote for gdb connection
Instead of deprecated 'target remote'.
Fix:
Warn : Prefer GDB command "target extended-remote :3333"
instead of "target remote :3333"

Signed-off-by: Julien Massot <julien.massot@iot.bzh>
2021-10-17 11:47:36 -04:00
Julien Massot
e4fac0e261 runners: openocd: gives configuration file before initialization
#38557 Reordered the argument for openocd, but we should not try
to call the init command before giving the board configuration
file stored in 'self.cfg_cmd'.

Move back this variable to it's original position.

Signed-off-by: Julien Massot <julien.massot@iot.bzh>
2021-10-17 11:47:36 -04:00
Julien Massot
1cac55cf3a runners: openocd do not try to iterate over gdb_init if None
Commits 49bcc08033 introduced
a possible None object 'gdb_init', do not try to iterate over
this variable if it is a None object.

Signed-off-by: Julien Massot <julien.massot@iot.bzh>
2021-10-17 11:47:36 -04:00
Peter Mitsis
ae394bff7c kernel: add support for event objects
Threads may wait on an event object such that any events posted to
that event object may wake a waiting thread if the posting satisfies
the waiting threads' event conditions.

The configuration option CONFIG_EVENTS is used to control the inclusion
of events in a system as their use increases the size of
'struct k_thread'.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2021-10-16 06:27:10 -04:00
Peter Mitsis
fc12f69cd0 scripts: Resolve python check warnings
Resolves two pylist check warnings that recommend using 'in'.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2021-10-16 06:27:10 -04:00
Dennis Ruffer
c714c78743 twister: Allow baud rates other than 115200
Fixes #38046
Document the changes in twister.rst
Add baud rate to debug message in twister
Add baud parameter to twister's add_device function
Set the twister baud rate from input parameters
Use 115200 as the default baud rate if not specified
Add baud to the hwmap-schema.yaml file
Add --device-serial-baud to twister arguments
Fix compliance issues
Fix mistake in parameter name from device-baud to device-serial-baud
Refactoring of the code in orded to simplify the logic
   and clear multiple definitions of the default baud rate.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
Signed-off-by: Dennis Ruffer <daruffer@gmail.com>
2021-10-14 16:35:48 -04:00
Henrik Brix Andersen
10fb5c203b runners: canopen: poll for flash ready
Poll the flash status instead of just reading the flash status once. Add
support for controlling the number of SDO retries and the SDO timeouts.

These changes allows for greater control of the CANopen program
download, which is especially useful on noisy or congested CAN networks
and on devices with slower flash access.

Fixes: #39409

Signed-off-by: Klaus H. Sorensen <khso@vestas.com>
Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
2021-10-14 10:44:21 +02:00
Carlo Caione
42105ed882 dts: Add support for specifier-space property.
Currently all the *-names and *-cells properties are derived from the
name of the base <name>s property. This is a limitation because:

- It forces the base property name to be plural ending in -s
- It doesn't allow the english exception of plural words ending in -es

With this patch we add one additional property 'specifier-space' that
can be used to explicitly specify the base property name.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Suggested-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-10-13 09:56:46 -05:00
Carles Cufi
6a3593f4af west: runners: Add a deprecation action to the core code
In order to allow for further options to be deprecated with minimal
impact, add a deprecation argparse Action and a callable instantiator
that can be used to deprecate options in favor of new ones.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2021-10-11 20:56:27 -04:00
Carles Cufi
2ccfd22594 runners: pyocd: Convert to -i/--dev-id
Remove the previous pyocd-specific --board-id option and switch to
the common -i/--dev-id one.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2021-10-11 20:56:27 -04:00
Carles Cufi
285c2cabf1 runners: dfu-util: Convert to -i/--dev-id
Remove the previous dfu-util-specific --pid option and switch to the
common -i/--dev-id one.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2021-10-11 20:56:27 -04:00
Carles Cufi
e3e23a51bb runners: canopen: Convert to -i/--dev-id
Remove the previous canopen-specific --node-id option and switch to the
common -i/--dev-id one.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2021-10-11 20:56:27 -04:00
Carles Cufi
5ee719ee9d runners: nrfjprog: Convert to -i/--dev-id
Remove the previous nrfjprog-specific --id option and switch to the
common -i/--dev-id one.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2021-10-11 20:56:27 -04:00
Carles Cufi
f6b13ddb6f runners: jlink: Convert to -i/--dev-id
Remove the previous jlink-specific --id option and switch to the common
-i/--dev-id one.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2021-10-11 20:56:27 -04:00
Carles Cufi
898f65f53c runners: Add new common -i/--dev-id command-line option
In an effort to standardize the way that a particular debugger or device
instance is identified when there are multiple present, introduce a new
-i/--dev-id option common to all runners that allows the user to specify
which device to interact with when there are multiple connected.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2021-10-11 20:56:27 -04:00
Sylvio Alves
49bcc08033 openocd: add hardcoded configs as input parameters
Debugging ESP32 SoCs is not possible if the commands
halt, target and load are enabled by default.
This MR allows this configs to be used as input parameters.

Signed-off-by: Sylvio Alves <sylvio.alves@espressif.com>
2021-10-08 12:59:55 +02:00
Daniel Leung
da9c40a7b8 scripts: footprint: fix print alignment and color
The size percentage was doing "xx.xx%" so 100% would not be
aligned correctly when printed. So add a space for percentage
printing.

Also change the coloring of "(hidden)" node so it would not
have the same color as files or symbols.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-10-07 16:56:20 -04:00
Daniel Leung
10eab23f08 west: sign/rimage: fix rimage argument if --no-manifest
If --no-manifest is specified, '-e' is still being passed
to rimage to build the extended manifest. Fix this so
that when --no-manifest is specified, '-e' is no longer
passed to rimage.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-10-06 19:43:30 -04:00
Martí Bolívar
a627666e06 device: add fudge factor for handle padding
When CONFIG_USERSPACE is enabled, the ELF file from linker pass 1 is
used to create a hash table that identifies kernel objects by address.
We therefore can't allow the size of any object in the pass 2 ELF to
change in a way that would change those addresses, or we would create
a garbage hash table.

Simultaneously (and regardless of CONFIG_USERSPACE's value),
gen_handles.py must transform arrays of handles from their pass 1
values to their pass 2 values; see the file's docstring for more
details on that transformation.

The way this works is that gen_handles.py just pads out each pass 2
array so its length is the same as its pass 1 value. The padding value
is a repeated run of DEVICE_HANDLE_ENDS values. This value is the
terminator which we look for at runtime in places like
device_required_handles_get(), so there must be at least one, and we
error out in gen_handles.py if there's no room in the pass 2 array for
at least one such value. (If there is extra room, we just keep
inserting extra DEVICE_HANDLE_ENDS values to pad the array to its
original length.)

However, it is possible that a device has more direct dependencies in
the pass 2 handles array than its corresponding devicetree node had in
the pass 1 array. When this happens, users have no recourse, so that's
a potential showstopper.

To work around this possibility for now, add a new config option,
CONFIG_DEVICE_HANDLE_PADDING, whose value defaults to 0.

When nonzero, it is a count of padding handles that are inserted into
each device handles array. When gen_handles.py errors out due to lack
of room, its error message now tells the user how much to increase
CONFIG_DEVICE_HANDLE_PADDING by to work around the problem.

It looks like a real fix for this is to allocate kernel objects whose
addresses are required for hash tables in CONFIG_USERSPACE=y
configurations *before* the handle arrays. The handle arrays could
then be resized as needed in pass 2, which saves ROM by avoiding
unnecessary padding, and would avoid the need for
CONFIG_DEVICE_HANDLE_PADDING altogether.

However, this 'real fix' is not available and we are facing a deadline
to get a temporary solution in for Zephyr v2.7.0, so this is a good
enough workaround for now.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-09-30 21:37:59 -04:00
Martí Bolívar
2c99173d64 Revert "device: simplify structure of handles array"
This reverts commit ec331c6fe2.

Although it's a valid simplification under the assumption that we're
going to be padding the array out anyway, it would use extra ROM if we
fix the build system issues that are currently forcing gen_handles.py
to introduce extra padding in the handles arrays for linker pass 2.

On the (perhaps optimistic) assumption that we're going to fix the
build system, let's get rid of a commit that would get in the way. The
extra "complexity" in device_required_handles_get() is trivial.

This gets rid of a comment describing the linker passes, but the
structure of the comment is a bit misleading (and it contains
incorrect information for the results of pass 2: the terminator at the
end is DEVICE_HANDLE_ENDS, not DEVICE_HANDLE_NULL).

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-09-30 21:37:59 -04:00
Martí Bolívar
288bcb90da Revert "device: iterable supported devices"
This reverts commit 0c6588ff47.

It's not clear that the supported devices are being properly computed,
so let's revert this for v2.7.0 until we've had more time to think
it through.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-09-30 21:37:59 -04:00
Piotr Golyzniak
0e1ecb373a twister: Fix missing testcases with error status
If timeout error was occured during performing testsuite on QEMU or
other simulator (e.g. Renode), information about this error will be not
placed in final report. It can be fixed, by set status of testcases,
which were not performed due to this error as "BLOCK".

Simmilar solution is alredy used in DeviceHandler when Twister works
with real platform. So I think that it is resonable to use the same
approach, when tests are performed on simulators.

I move loop which iterate through all testcases and set their tests
status as BLOCK into the parent Handler class, because the same
activity is performed in each children class (BinaryHandler,
DeviceHandler and QEMUHandler).

Fixes #38756

Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
2021-09-28 21:34:10 -04:00
Jaxson Han
f079e6638f scripts: Add align for empty _app_smem region
When _app_smem region is empty, alignment is also needed. If there
is no alignment, the _app_smem_start used by arm mpu can be lower
than __rodata_region_end, and this two regions can overlap.

The Armv8-R aarch64 MPU does not allow overlapped regions.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2021-09-28 20:06:06 -04:00
Alex Sergeev
e7778b8584 net: ptp: extract PTP interfaces outside gPTP subsys under NET_L2_PTP
As per #38352, we would like to start building out PTP (IEEE 1588)
support for superset of gPTP functionality in Zephyr. This is the first
step to abstract away some key interfaces from NET_GPTP umbrella to
NET_L2_PTP.

Signed-off-by: Alex Sergeev <asergeev@carbonrobotics.com>
2021-09-28 20:01:16 -04:00
Torsten Rasmussen
af87599e11 scripts: twister: print message from CMake verify toolchain on failures
Fixes: #38924

When the `verify-toolchain.cmake` script fails, then twister will print
a standard message to the user regardless of the reason.
> E: Variable ZEPHYR_TOOLCHAIN_VARIANT is not defined

The `verify-toolchain.cmake` already prints detailed information
regarding the cause of the failure, so twister should just pass that
message as-is.

For example, the message that is provided by verify-toolchain.cmake
when Zephyr SDK 0.13.0 is installed but 0.13.1 is required looks like:
> CMake Error at cmake/verify-toolchain.cmake:75 (find_package):
>  Could not find a configuration file for package "Zephyr-sdk" that is
>  compatible with requested version "0.13.1".
>
>  The following configuration files were considered but not accepted:
>
> /opt/zephyr-sdk-0.13.0/cmake/Zephyr-sdkConfig.cmake, version: 0.13.0

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-09-28 14:10:37 -04:00
Iuliana Prodan
3ec07c5f47 west: sign: add support for NXP i.MX8QXP board
Add support for signing i.MX8QXP SOF with Zephyr images with rimage.

i.MX8QXP and i.MX8QM have the same board-level definitions,
so we use the generic imx8.

Signed-off-by: Iuliana Prodan <iuliana.prodan@nxp.com>
2021-09-27 21:52:31 -04:00
Henrik Brix Andersen
9aa4a70e5c twister: ignore ROM region overflows
Add ROM region to list of regions to optionally ignore overflows on.

Signed-off-by: Henrik Brix Andersen <henrik@brixandersen.dk>
2021-09-27 17:53:09 -04:00