Commit Graph

2743 Commits

Author SHA1 Message Date
Torsten Rasmussen
ab7ec17b86 cmake: zephyr module processing improvements.
This commit improves Zephyr modules support in CMake.
All Zephyr modules will now have a corresponding
`ZEPHYR_${module_name}_MODULE_DIR` set, regardless of whether they
define a cmake folder in module.yml.

This results in:
`ZEPHYR_${module_name}_MODULE_DIR` defined for all modules and refers to
the root of the modle.
`ZEPHYR_${module_name}_CMAKE_DIR` defined for modules that specifies
cmake in module.yml, or is having both a CMakeLists.txt and Kconfig file
in the zephyr folder.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-08-31 12:15:02 +02:00
Torsten Rasmussen
25c3f8e12f scripts: zephyr_modules.py cleanup.
This commit has no functional changes but is simply a cleanup so that
flake8 will pass without warnings.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-08-31 12:15:02 +02:00
Anas Nashif
9eb9c4cce9 sanitycheck: add option --emulation-only
This will only build/run on emulation platforms.
The decision is made based on the value of the 'simulation' key in the
platform yaml file.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-08-28 14:22:07 -07:00
Øyvind Rønningstad
7012fdf0b7 nrfjprog.py: Fix snr globbing to allow leading 0s
This is needed for sanitycheck hardware maps which take the serial
number directly from USB metadata.

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2020-08-28 11:38:05 +02:00
Anas Nashif
dca317c730 sanitycheck: inclusive language
change whitelist -> allow.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-08-27 07:04:07 -04:00
Daniel Leung
faae15d838 sanitycheck: allow QEMU to crash without failing
Adds some code to allow QEMU to crash without failing the test.
This is required for testing coredump code as it will certainly
cause QEMU to crash.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-08-24 20:28:24 -04:00
Daniel Leung
5b1b4a3755 sanitycheck: need to wait for harness to process QEMU outputs
There is a potential that QEMUHandler.handle() returns before
the testing harness finishes processing all the output from QEMU.
Simply wait for the harness thread to finish before returning.

Also, fix the return code in the debug message as it should be
the return code from Popen().

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-08-24 20:28:24 -04:00
Daniel Leung
181d07321f coredump: add support for ARM Cortex-M
This adds the necessary bits in arch code, and Python scripts
to enable coredump support for ARM Cortex-M.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-08-24 20:28:24 -04:00
Daniel Leung
8fbb14ef50 coredump: add support for x86 and x86_64
This adds the necessary bits to enable coredump for x86
and x86_64.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-08-24 20:28:24 -04:00
Daniel Leung
49206a86ff debug/coredump: add a primitive coredump mechanism
This adds a very primitive coredump mechanism under subsys/debug
where during fatal error, register and memory content can be
dumped to coredump backend. One such backend utilizing log
module for output is included. Once the coredump log is converted
to a binary file, it can be used with the ELF output file as
inputs to an overly simplified implementation of a GDB server.
This GDB server can be attached via the target remote command of
GDB and will be serving register and memory content. This allows
using GDB to examine stack and memory where the fatal error
occurred.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-08-24 20:28:24 -04:00
Daniel Leung
203556cd8c sanitycheck: update section names in size calculator
Commit 8d7bb8ffd8 refactored
device structures which changed some of the linker sections:
"devconfig" was removed, and "devices" was added. However,
the list in sanitycheck's size calculator was not updated,
which results in sanitycheck complaining about unrecognized
sections when doing footprint.

Also, a few sections have been renamed (with added suffix
"_area") due to introduction of Z_ITERABLE_SECTION_RAM/ROM
macros. There are also some missing section names.

Fixes the issue by adding the missing sections names, and
updating existing ones.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-08-24 20:18:06 -04:00
Anas Nashif
5ab117ffd2 tracing: scripts: add scripts for parsing ctf data
Generate trace using samples/subsys/tracing for example:

    west build -b qemu_x86 samples/subsys/tracing  -t run \
      -- -DCONF_FILE=prj_uart_ctf.conf

    mkdir ctf
    cp build/channel0_0 ctf/
    cp subsys/tracing/ctf/tsdl/metadata ctf/
    ./scripts/tracing/parse_ctf.py -t ctf

1969-12-31 19:00:00.001779 (+0.000000 s): thread_create: thread_a
1969-12-31 19:00:00.001915 (+0.000136 s): thread_info (Stack size: 1024)
1969-12-31 19:00:00.002021 (+0.000107 s): thread_ready: thread_a
1969-12-31 19:00:00.002287 (+0.000265 s): thread_switched_out: main
1969-12-31 19:00:00.002320 (+0.000033 s): thread_switched_in: main
1969-12-31 19:00:00.002585 (+0.000265 s): thread_abort: main
1969-12-31 19:00:00.002730 (+0.000145 s): thread_switched_out: main
1969-12-31 19:00:00.002762 (+0.000032 s): thread_switched_in: thread_a
1969-12-31 19:00:00.002829 (+0.000067 s): thread_create: 1130656
1969-12-31 19:00:00.002860 (+0.000031 s): thread_info (Stack size: 1024)
1969-12-31 19:00:00.002911 (+0.000052 s): thread_ready: 1130656
1969-12-31 19:00:00.003033 (+0.000121 s): thread_name_set
1969-12-31 19:00:00.003132 (+0.000100 s): semaphore_take (1140992)
1969-12-31 19:00:00.003201 (+0.000069 s): end_call 38 (SEMA_TAKE)
1969-12-31 19:00:00.003330 (+0.000128 s): start_call 39 (SLEEP)
1969-12-31 19:00:00.003721 (+0.000391 s): thread_suspend: thread_a
1969-12-31 19:00:00.003802 (+0.000081 s): thread_switched_out: thread_a
1969-12-31 19:00:00.003834 (+0.000032 s): thread_switched_in: thread_b
1969-12-31 19:00:00.003872 (+0.000038 s): semaphore_take (1141016)
1969-12-31 19:00:00.003976 (+0.000103 s): thread_pending: thread_b
1969-12-31 19:00:00.004095 (+0.000119 s): thread_switched_out: thread_b
1969-12-31 19:00:00.004127 (+0.000032 s): thread_switched_in: idle 00
1969-12-31 19:00:00.004232 (+0.000105 s): idle
1969-12-31 19:00:00.510848 (+0.506616 s): isr_enter
1969-12-31 19:00:00.511664 (+0.000816 s): thread_resume: thread_a
1969-12-31 19:00:00.511947 (+0.000283 s): thread_ready: thread_a
1969-12-31 19:00:00.512385 (+0.000438 s): isr_exit
1969-12-31 19:00:00.512570 (+0.000186 s): thread_switched_out: idle 00
1969-12-31 19:00:00.512689 (+0.000119 s): thread_switched_in: thread_a
1969-12-31 19:00:00.512869 (+0.000180 s): end_call 39 (SLEEP)

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-08-24 13:21:12 +02:00
Fabio Utzig
716ab47dcc west: try image-1's size as slot size in imgtool
When MCUBoot is built using the swap-move strategy, the secondary slot
can be one sector smaller than the primary slot, because the primary
slot's last "useful" image sector must be "reserved" for the move
operation; this impacts the generation of padded images, so when
signing an image, the proper size to use is the secondary slot's,
unless a secondary image is not defined which is the case when using
single image DFU.

Signed-off-by: Fabio Utzig <fabio.utzig@nordicsemi.no>
2020-08-24 10:18:56 +02:00
Wenbo Yang
17216128ab hardening: remove the empty line before EOF
The extra empty line broke the "ninja hardenconfig" on my machine with
Python 3.7.5, it complains:

"
... ...
File "/home/zephyrproject/zephyr/scripts/kconfig/hardenconfig.py",
line 46, in compare_with_hardened_conf

name = row[0]
IndexError: list index out of range
FAILED: CMakeFiles/hardenconfig
"

The csv.reader reads this empty line and gets an empty list which will
not be successfully "de-referenced".  Removing it to improve the
out-of-box experience when pepople try out the hardening options.

Signed-off-by: Wenbo Yang <wenbo.yangcn@gmail.com>
2020-08-21 15:09:07 -04:00
Peter A. Bigot
e49e3c8fef scripts: update checkpatch based on current linux kernel version
This squashes and applies the Linux diffs to scripts/checkpatch.pl
between Linux commit 16fbf79b0f83bc75 ("Linux 5.6-rc7") and
9123e3a74ec7b93 ("Linux 5.9-rc1") except for commits identified below.

The last 1000 commits to Zephyr master were compared for checkpatch
output differences between the previous Zephyr version and this
version.  One new diagnostic about function declarations with an empty
parameter-list was introduced (FUNCTION_WITHOUT_ARGS).  The text of an
existing diagnostic was changed (DT_SPLIT_BINDING_PATCH).  The text of
LONG_LINE diagnostics was enhanced to provide the actual line length.

Linux commit dfa05c28ca7ffc0a ("checkpatch: remove email address
comment from email address comparisons") was removed because
differences in the scripts resulted in false signed-off-by check
diagnostics when a full name included characters not in Basic Latin,
due to changes in how the author name was extracted.  Earlier upstream
changes not integrated into Zephyr may be required.

Linux commit b95692f8b3000166 ("checkpatch: prefer fallthrough; over
fallthrough comments") was removed because Zephyr doesn't support the
upstream pseudo keyword.

Linux commit bdc48fa11e46f867 ("checkpatch/coding-style: deprecate
80-column warning") was edited to not actually change the 80-column
maximum line limit as this change has not been mooted for Zephyr.

Linux commit ced69da1db0b57bb ("checkpatch: fix CONST_STRUCT when
const_structs.checkpatch is missing") was edited to the CONST_STRUCT
file as that's not supported in Zephyr.

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2020-08-21 13:57:02 -04:00
Henrik Brix Andersen
96a39f00c2 west: runners: jlink: add support for -nogui 1 command line parameter
Add support for the J-Link Commander "-NoGui 1" command line parameter
in the West J-Link runner.

This command line parameter suppresses GUI dialogs (except for license
dialogs) in J-Link Commander starting from v6.80.

Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
2020-08-20 13:04:37 -05:00
Martí Bolívar
2ed2d3bfa9 scripts: runners: add --no-erase support
If a runner supports the --erase option, make it so that the user can
add --no-erase to the command line to explicitly disable it.

Add a diagnostic at info() level whenever this option is requested.

The intent of this commit is to make it more obvious when a mass erase
was requested, especially in situations (like MCUboot with
CONFIG_MCUBOOT_TRY_MASS_ERASE=y) where this option may be turned on by
default.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-08-19 16:38:26 -05:00
Torsten Rasmussen
edde894d04 cmake: Zephyr CMake package clean-up and minor fix
Fixes: #27375

This is a cleanup of the Zephyr CMake package export.
The code has been simplified so that the export now happens through a
CMake script. This avoids several generated CMake build files compared
to previous export mode, and thus removes the need for a CMake pristine
script.

A benefit of this cleanup is that it also fixes #27375.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-08-19 12:52:33 -04:00
Maciej Perkowski
c67a0cd098 sanitycheck: Improve tests counting
This patch should improve couting and reporting of the number
of passed/skipped tests.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2020-08-19 12:49:42 -04:00
Martí Bolívar
112e1d32b2 scripts: west_commands: remove unused helper
The cached_runner_config() helper is no longer used, so remove it.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-08-19 09:51:13 -04:00
Martí Bolívar
7492997ba7 scripts: west sign: clean up and address TODOs
Avoid use of the cached_runner_config() helper, which we have a TODO
item to get rid of. Adjust the output and do some other minor
improvements.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-08-19 09:51:13 -04:00
Martí Bolívar
9ca1de18eb scripts: west_commands: add .config parsing helper
Rolling yet another parser turns out to be the best way to let west
extensions respond to Kconfig values. See source code comments in the
patch for details.

The sanitylib library has some similar functionality but it isn't
exactly the same, and it gets strings wrong. For example, that parser
can't handle this option:

CONFIG_FOO="he said \"no\" to me"

This one can, and it has a couple of other features we'll find useful
for west extensions eventually besides.

(Not to mention that sanitylib also rolled its own CMake cache parser,
which also exists in west_commands.)

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-08-19 09:51:13 -04:00
Anas Nashif
74f4891a8f requirements: add anytree
Needed now by both ram and rom report targets.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-08-14 18:49:26 -04:00
Anas Nashif
239d175eff scripts: size_report: fix path prepending
Fixes potential issue on windows.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-08-14 18:49:26 -04:00
Anas Nashif
ac215a570c size_report: use anytree
Use anytree module to display tree and to allow easy exporting into
json.
Add option to export results into a json file.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-08-14 18:49:26 -04:00
Øyvind Rønningstad
6cc7ea51e4 nrfjprog.py: Allow passing a glob to --snr
You can now do --snr 6*1* to achieve the same as --snr 683010000

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2020-08-14 13:32:36 +02:00
Øyvind Rønningstad
3b595ebd3e nrfjprog.py: Small refactor of get_board_snr()
Move some functionality out into separate functions for clarity.

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2020-08-14 13:32:36 +02:00
Wayne Ren
938642cf7f scripts: add the support of DW_AT_count for array type
In DWARF 4, e.g. ARC Metaware toolchain, DW_AT_count is
used not DW_AT_upper_bound. We should consider this corner
case.

Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
2020-08-14 13:30:41 +02:00
Anas Nashif
d548a890b8 ci: do not run sanitycheck on MAINTAINERS.yml changes
Skip sanitycheck if we are only changing MAINTAINERS.yml

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-08-13 17:03:47 +02:00
Kumar Gala
cfc54f38be sanitycheck: set selected_plaforms for --test-only
When running with --test-only we get:

Traceback (most recent call last):
  File "./scripts/sanitycheck", line 1168, in <module>
    main()
  File "./scripts/sanitycheck", line 1160, in main
    options.only_failed)
  File "scripts/sanity_chk/sanitylib.py", line 2543, in save_reports
    self.xunit_report(filename + ".xml", full_report=False, append=only_failed)
  File "scripts/sanity_chk/sanitylib.py", line 3220, in xunit_report
    return fails, passes, errors, skips
UnboundLocalError: local variable 'fails' referenced before assignment

This is due to the fact that selected_platforms was not set.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-08-13 07:45:17 -05:00
Christian Taedcke
b2be804421 sanitycheck: Fix xml testsuite attribute skipped
In the junit output the attribute containing the number of skipped tests
must be named "skipped" instead of "skip".
See e.g. https://github.com/junit-team/junit5/blob/main/platform-tests/
src/test/resources/jenkins-junit.xsd#L95

Signed-off-by: Christian Taedcke <christian.taedcke@lemonbeat.com>
2020-08-13 11:51:19 +02:00
Wenbo Yang
999290278e hardening: Deal with empty lines in csv file
The extra empty line broke the "ninja hardenconfig" on my machine with
Python 3.7.5, it complains:

"
... ...
File "/home/zephyrproject/zephyr/scripts/kconfig/hardenconfig.py",
line 46, in compare_with_hardened_conf

name = row[0]
IndexError: list index out of range
FAILED: CMakeFiles/hardenconfig
"

The csv.reader reads this empty line and gets an empty list which will
not be successfully "de-referenced". Adding extra check to skip the
empty lines.

Signed-off-by: Wenbo Yang <wenbo.yangcn@gmail.com>
2020-08-12 16:43:18 -04:00
Peter Bigot
78f36fa534 python3: add check_compliance package requirements
Running check_compliance on a PR before submitting it can avert
embarrassing mistakes.  Ensure the packages needed to do so are
installed along with all the others.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2020-08-11 14:49:48 -04:00
Marcin Niestroj
2652dc71a3 sanitycheck: fix --coverage-tool gcovr
Currently '--coverage-tool gcovr' results in using lcov. Fix that to use
gcovr, as requested by user.

Fixes: f6462a3a8c ("sanitycheck: get rid of global VERBOSE")
Signed-off-by: Marcin Niestroj <m.niestroj@grinn-global.com>
2020-08-11 08:36:39 -04:00
Maciej Perkowski
e3ff4cfcd6 sanitycheck: tests: Fill results for skipped tests
Sanitycheck discards test instances if a set of given coditions
is not fulfilled. This leads to empty test results for these
instances. This can introduce ambiguity with results that are
missing due to some bugs in the framework. This commit fills
the results for skipped tests with 'skipped' states and provides
the reason for filtering them out in the msg field. The commit
also fixes the way sanitycheck counts and reports tests/test cases

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2020-08-06 11:49:39 +02:00
Torsten Rasmussen
25d57ba5af modules: adding support for ROOTs definitions in zephyr/module.yml
Fixes: #25215

This commit introduces the possibility of defining ROOTs in a Zephyr
module and have it automatically appended to list of other ROOTs.
Supported with this commit:
- BOARD_ROOT
- SOC_ROOT
- DTS_ROOT
- ARCH_ROOT

In order to support this in Zephyr module files, the detection of west
has been moved to dedicated west.cmake file and included immediately
after python.cmake.

Also the inclusion of zephyr_modules.cmake has moved before first use
of BOARD_ROOT.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-08-05 08:06:07 -04:00
Torsten Rasmussen
5f7cc8ded9 cmake: kconfig: support for multiple SOC_ROOT
This commit introduces support for multiple SOC_ROOT.

This means that additional SOC_ROOTs specified using -DSOC_ROOT as
argument to CMake will be forming a list together with ${ZEPHYR_BASE}.

This allows for greater flexibility, as developers can now specify
multiple out-of-tree SoCs and not worry about the SoC used for the
board they compile for.

Also it avoid code, such as:
if(BOARD STREQUAL my_board_using_out_of_tree_soc)
  set(SOC_ROOT some/out/of/tree/soc/path)
endif()
in application CMakeLists.txt.

Finally, allowing multiple SOC_ROOTs prepares for specifying SOC_ROOTs
in Zephyr modules.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-08-05 08:06:07 -04:00
Øyvind Rønningstad
f72aef15d2 sanitycheck: Make --hardware-map and --west-flash together
When passing a using --hardware-map, sanitycheck would ignore extra
args passed via --west-flash, such as --west-flash="--erase".

This is because the command with the extra args was overwritten by the
command with the runner info from the hardware map.

This patch merges those code paths so they are aware of each other and
of --west-runner.

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2020-08-03 20:40:50 -04:00
Maciej Perkowski
e7f4074799 sanitycheck: Modify expected_subcases
Before sanitycehck was removing all `test_` chars in test cases'
names. This test has to be modify to work with the improved behavior

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2020-08-03 19:46:15 -04:00
Maciej Perkowski
034d4f2156 sanitycheck: Fix test cases names
This commit limits the removal of `test_` from test case name
to only the first occurance. There are test cases with `test_`
also in the middle of their names and removing it couses mismatches
between extracted testcases and the names obtained when passing

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2020-08-03 19:46:15 -04:00
Andrei Emeltchenko
fb44188ebb west: sign: Move imgtool specific docs to appropriate place
Move documentation specific to imgtool to appropriate place.

Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
2020-08-03 13:49:17 +02:00
Andrei Emeltchenko
51182ab881 west: sign: Add rimage signing tool documentation
Add documentation specific to rimage signing tool.

Fixes #26596

Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
2020-08-03 13:49:17 +02:00
Andrew Boie
8ce260d8df kernel: introduce supervisor-only stacks
These stacks are appropriate for threads that run purely in
supervisor mode, and also as stacks for interrupt and exception
handling.

Two new arch defines are introduced:

- ARCH_KERNEL_STACK_GUARD_SIZE
- ARCH_KERNEL_STACK_OBJ_ALIGN

New public declaration macros:

- K_KERNEL_STACK_RESERVED
- K_KERNEL_STACK_EXTERN
- K_KERNEL_STACK_DEFINE
- K_KERNEL_STACK_ARRAY_DEFINE
- K_KERNEL_STACK_MEMBER
- K_KERNEL_STACK_SIZEOF

If user mode is not enabled, K_KERNEL_STACK_* and K_THREAD_STACK_*
are equivalent.

Separately generated privilege elevation stacks are now declared
like kernel stacks, removing the need for K_PRIVILEGE_STACK_ALIGN.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-07-30 21:11:14 -04:00
Kumar Gala
33056e48c5 ci: make test_file.txt a build artifact for debug
Rather that echo'ng the test_file.txt to the console, lets just upload
it as a buildkite artifact.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-07-30 11:25:25 -04:00
Maciej Perkowski
732ab5311d doc: Fix path to requirements
Fix path to requirements

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2020-07-30 16:40:29 +02:00
Anas Nashif
555fc6dbd5 sanitycheck: fix only_tags usage
We have been doing an AND comparison instead of an OR. AND does exclude
way to many testcases where multiple tags are being used.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-30 10:30:23 -04:00
Anas Nashif
314b8af797 ci: enable sanitycheck for integration
Enable sample for integration.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-30 08:00:03 -04:00
Anas Nashif
1636c316d9 sanitycheck: introduce integration testing
Use --integration testing for CI to limit execution and builds on
platforms that actually provide most of the coverage instead of blindly
building/running on all available platforms.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-30 08:00:03 -04:00
Anas Nashif
1bf1d19a42 sanitycheck: realign schema
Just a cleanup and alignment of the yaml file.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-30 08:00:03 -04:00
Simon Guinot
2dad0b4500 scripts/requirements: add lpc_checksum script
On some NXP LPC MCUs (such as LPC11U6x), a valid firmware image must
include a checksum in the 7th entry of the ARM Cortex-M0 exception
vector table. This checksum is verified by the bootloader.

The lpc_checksum.py python script allows to insert this checksum into
the firmware images (BIN and HEX formats).

Project page: https://github.com/basilfx/lpc_checksum

Signed-off-by: Simon Guinot <simon.guinot@seagate.com>
2020-07-29 20:12:24 +02:00
Jukka Rissanen
bfa08cd04d userspace: Make network interface a kernel object
Make net_if a kernel object with type K_OBJ_NET_IF so that we
can restrict access to it.

Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
2020-07-29 08:13:22 -04:00
Anas Nashif
f8c024a22f sanitycheck: remove verbose log message
This was added for debugging and was left in the script by mistake.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-27 19:03:10 +02:00
Anas Nashif
1c2f127e3f sanitycheck: do not duplicate code in tests
Fixed tests to use library code instead of duplicating logic in the test
itself.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-27 11:58:32 +02:00
Anas Nashif
9e1be4c05e sanitycheck: do not expect results from build_only instances
instances that do not run will have no results (beside the fact they
built successfully), so log those as such.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-27 11:58:32 +02:00
Martí Bolívar
469f53c210 scripts: sanitycheck: import edtlib
This is a cargo-culted attempt to make an error observed in poorly
understood CI circumstances go away when loading edt.pickle, by making
edtlib visible in sys.modules before loading the pickle file.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-24 17:58:03 -04:00
Wentong Wu
517633c101 scripts: sanitylib.py: add exception check for psutil
Add exception check for psutil when running CI with QEMU.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2020-07-24 14:26:33 -04:00
Anas Nashif
8d8875b752 scripts: add maintainer script
Adding the get_maintainer.py script

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2020-07-24 17:07:48 +02:00
Kumar Gala
feeda05323 ci: Add a debug out of what tests we are going to build
Add a cat of the test_file.txt so we can see exactly what tests are
expected to be built and run.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-07-24 12:10:15 +02:00
Martí Bolívar
9861e5d22a sanitylib: log test exception tracebacks
This aids in debugging.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-23 13:36:22 -05:00
Martí Bolívar
9c92baa73f devicetree: use edt.pickle more
Consolidate creation of edtlib.EDT objects from a build directory's
devicetree into one place by loading it from build/zephyr/edt.pickle
everywhere. A previous commit creates edt.pickle from gen_defines.py.

In addition to probably speeding things up slightly by not reparsing
the devicetree, the main benefit of this approach is creating a single
point of truth for the bindings directories and warnings
configuration, meaning we don't have to worry about them getting out
of sync while being passed around between devicetree creation and
usage time.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-23 09:19:45 -05:00
Jan Van Winkel
716eea0b65 scripts: Increased minimal pyelftools version
Increased minimal pyelftools version to 0.26 else
scripts/footprint/size_report will fail to import LocationExpr from
elftools.dwarf.locationlists

Signed-off-by: Jan Van Winkel <jan.van_winkel@dxplore.eu>
2020-07-23 07:49:35 -04:00
Martí Bolívar
006319f043 scripts: kconfigfunctions: add a comment
I can never remember what the contents of the kconfigfunctions dict is
and have to look it up every time. I suspect I'm not alone, and not
everyone who uses these will know how to look up the answer. Add a
comment explaining the value.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-23 11:00:39 +02:00
Martí Bolívar
24677f93bc kconfig: add dt_path_enabled, dt_alias_enabled
There is already a way to check if a node is enabled based on its node
label, but we don't have an equivalent way to do that for a path or an
alias. Add them. These rely on the same underlying edtlib machinery to
get their jobs done, but are being treated differently within Kconfig
in order to match distinctions between paths and aliases made in the
devicetree.h API.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-23 11:00:39 +02:00
Luiz Augusto von Dentz
75f47a56b5 Bluetooth: Add config option to disable security checks
This adds CONFIG_BT_CONN_DISABLE_SECURITY which can be used to disable
security checks for incoming requests enabling to test accessing GATT
attributes and L2CAP channels that would otherwise require
encryption/authentication in order to be accessed.

It depends on BT_TESTING to indicate to the users that this is a
testing feature which shall not be used in production.

Signed-off-by: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
2020-07-23 11:00:04 +02:00
Maureen Helm
8c7cef6da0 runners: Implement --erase option in pyocd runner
Implements the common runner option to mass erase before flashing in the
pyocd runner.

Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
2020-07-23 10:30:47 +02:00
Anas Nashif
e47866c530 sanitycheck: fix --device-testing without serial-pty
serial-pty is an extension that is not supported in the hardware map, so
we need to work around the fact that it is not part of the map when
dealing with normal serial.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-22 16:57:14 -05:00
Andrei Emeltchenko
d8b845be4e sanitycheck: Add serial_pty script command line option
Add option to provide to sanitycheck argument for creating
pseudoterminal. Used with hardware without serial console connected.
A pseudoterminal is used to make a sanitycheck believe that it
interacts with a terminal although it actually interacts with the
script.

E.g "sanitycheck --device-testing --device-serial-pty <script>"

Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
2020-07-21 20:19:33 -04:00
Anas Nashif
e8e367ada4 sanitycheck: support only_tags for boards
Support running/building only specific tags and ignoring everything
else.

Adding this to a platform yaml file will enable tests with one of of
tags listed to be executed on the platform.

This is useful for special platform configurations used for testing
specific features for example.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-20 17:44:16 -04:00
Ruslan Mstoi
6f26ca0b60 scripts: tests: Add tests for subfolder_list.py
This commit implements tests of subfolder_list.py

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-07-20 17:31:53 -04:00
Ruslan Mstoi
ab20641054 scripts: subfolder_list.py: refactor the code
Refactor subfolder_list.py to enable unit testing. This means split the
implementation into separate logical functions. Additionally, modify the
code to improve PEP 8 compatibility and pylint result.

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-07-20 17:31:53 -04:00
Carles Cufi
02dea9279b scripts: modules: Fix initialization of variable
The projects variable needs to be initialized with the list of modules
provided via cmd-line arguments.

This is a regression introduced by
ef3c5e5516.

Fixes #26948.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2020-07-20 22:39:09 +02:00
Anas Nashif
5092541e2b sanitycheck: disable erroring on warnings
--warnings-as-errors was basically doing nothing, it is the default.
Replace this with an option to disable erroring on warning:

 -W, --disable-warnings-as-errors

Fixes #26910

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-17 13:03:36 -05:00
Anas Nashif
06052927be sanitycheck: capture fails correctly on qemu crashes
We still have cases where qemu crashes are not being correctly captured
in the on-screen summary, however they are captured in the final
generated report.

Fixes #26679

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-16 15:44:17 -04:00
Daniel Leung
ba5f627815 scripts: size_report: rework to use pyelftools...
...instead of GNU binutils to extract DWARF information.
This is now a bit more portable across OS and toolchain.
One bouns is that this nows with on qemu_x86_64.

A few differences are:
() z_mrsh_* which are aliases to handler_no_syscalls() are now
   dropped as they are mapped to the same address, so that they
   are not counted multiple times.
() Static functions and variables with same names should now be
   attributed to the correct source files instead of being
   accumulated into the same symbol of one file (e.g. multiple
   thread_entry() in kernel tests).
() The totals for ROM and RAM are calculated from the
   corresponding sections. Previous script includes the debug
   sections as total ROM size which is not entirely correct.

Fixes #22996

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-07-16 12:14:22 +02:00
Spoorthy Priya Yerabolu
0e599be5fc sanitycheck: Updating README file
Updated readme with coverage commands and
organization of tests

Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
2020-07-09 16:03:57 -05:00
Anas Nashif
dc43c298a2 sanitycheck: do not retry build errors with --retry-failed
restore how --only-failed works by allow rebuilds in case of build
errors, however, do not rebuild when --retry-failed is used, which is a
CI usecase and nothing would change in the second iteration.

Fixes #26685

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-09 16:46:24 -04:00
Stephanos Ioannidis
3322489d22 config: Rename TEXT_SECTION_OFFSET to ROM_START_OFFSET
The `TEXT_SECTION_OFFSET` symbol is used to specify the offset between
the beginning of the ROM area and the address of the first ROM section.

This commit renames `TEXT_SECTION_OFFSET` to `ROM_START_OFFSET` because
the first ROM section is not always the `.text` section.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2020-07-09 14:02:38 -04:00
Christian Taedcke
3dbe9f2960 sanitycheck: native: Add option to enable UBSAN
Add option for native platform to enable undefined behaviour sanitizer.

Signed-off-by: Christian Taedcke <christian.taedcke@lemonbeat.com>
2020-07-09 07:49:57 -04:00
Aastha Grover
cb0f65e410 requirements-build-test.txt: Updating mock module version
Updating mock module version to >=4.0.1 as sanitycheck testsuite
might fail in some systems due to old version.

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2020-07-09 06:26:28 -04:00
Anas Nashif
b8c7de6d9a actions: update daily version for testing
Update versions.json with the daily commit that device testing will be
performed on.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-09 11:29:56 +02:00
Torsten Rasmussen
ef3c5e5516 cmake: west: invoke west using same python as rest of build system
When running CMake, then Python3 will be used.
This is detected through FindPython3, with a preference for using the
python or python3 in path, if any of those matches the required Python
minimal version in Zephyr.

It is also possible for users to specify a different Python, as example
by using:
`cmake -DPYTHON_PREFER=/usr/bin/python3.x`

However, when running `west` as native command, then west will be
invoked on linux based on the python defined in:
`west` launcher, which could be: `#!/usr/bin/python3.y`

Thus there could be mismatch in Pythons used for `west` and the python
used for other scripts.

This is even worse on windows, where a user might experience:
```
>.\opt\bin\Scripts\west.exe --version
Traceback (most recent call last):
  File "C:\Python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  ...
  File "C:\Python37\lib\socket.py", line 49, in <module>
    import _socket
ImportError: Module use of python38.dll conflicts with this version of
Python.
```

when testing out a newer Python, but the python in path is still a 3.7.

By importing `west` into zephyr_module.py and by using, as example
`python -c "from west.util import west_topdir; print(topdir())"`
we ensure the same python is used in all python scripts.

Also it allows the user to control the python to use for west.

It also ensures that the west version being tested, is also the version
being used, where old code would test the version imported by python,
but using the west in path (which could be a different version)

If the west version installed in the current Python, and west invocation
is using a different Python interpreter, then an additional help text
is printed, to easier assist users with debugging.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-07-08 23:23:04 -04:00
Kumar Gala
4e2988deeb edtlib: Fold _binding_compat into _init_compat2binding
We only have one use of _binding_compat and it doesn't need self, so
just fold it into _init_compat2binding.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-07-08 22:03:24 -04:00
Kumar Gala
6bf761fc0a dts: Remove support for deprecated DTS binding syntax
We deprecated a number of aspects of the DTS binding syntax in Zephyr
2.1.  Remove the support for the deprecated syntax.  Remove from docs
about the deprecated syntax as well.

Removed reference in release-notes-2.1.rst to legacy_binding_syntax
since that anchor doesn't exist anymore.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-07-08 22:03:24 -04:00
Martí Bolívar
d8698cbdc6 sanitylib: pacify pylint
According to our compliance checks, we have 'useless-suppression'
and 'unused-import' on our hands:

W0611: Unused CLoader imported from yaml as Loader
W0611: Unused Loader imported from yaml
I0021: Useless suppression of 'unsubscriptable-object'

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-08 20:19:02 -04:00
Martí Bolívar
148542c080 build: remove dead stores to environment
Now that kconfigfunctions.py is loading the EDT object from a pickle
file, I can't find any other reads of the environment variables it was
previously using to do that.

(The CMake variables with the same names are read in different places;
I'm specifically talking about environment variables here.)

Remove the dead stores.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-08 14:02:44 -05:00
Martí Bolívar
269f350487 kconfig: load edt from edt.pickle
This saves time and avoids the need to pass additional state around in
the environment to recreate the edt exactly.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-08 14:02:44 -05:00
Martí Bolívar
533f451e7e dts: marshal the EDT object for later use
We need to save and restore the devicetree data to generate optimized
dependency information later on in the build, in particular during the
final application link.

Make this happen by pickling the EDT object in BUILD_DIR/edt.pickle.

The existence of this file is an implementation detail, so do not add
it to the documentation.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-08 14:02:44 -05:00
Martí Bolívar
e76b7205ef dts: edtlib: allow pickling/unpickling EDT objects
We have a use case for saving the EDT object to be able to open it up
again later. It would be convenient to be able to do this with the
pickle module from stdlib.

The only thing stopping us from doing that appears to be the open
reference to sys.stderr that's held the edt object even after
EDT.__init__ exits. However, there doesn't seem to be a need to keep
holding on to this object, and in fact it would be a little bit nicer
to drop the reference in case something else (even in the same Python
process that created it originally) wants the EDT object around, but
might want the warn file closed if its refcount zeroes out.

Just drop the reference at the end of __init__ and make EDT._warn()
throw an exception if it's attempted to be used after the constructor
exits.

Make pickle-ability an API guarantee so we can treat any regressions
as bugs going forward.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-08 14:02:44 -05:00
Martí Bolívar
e05c94e334 dts: edtlib: fix reg / unit addr warn location
I can't see a good reason to be doing this in the Node class's
unit_addr accessor. Move it up to the edtlib initialization so it only
happens once.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-08 14:02:44 -05:00
Anas Nashif
ae61b7edd6 sanitycheck: Use the C LibYAML parser if available
Use the C LibYAML parser if available, rather than the Python parser. It
is much faster.

This is a clean and rebased version of PR #20210 by
Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>

Co-authored-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-08 13:34:45 -05:00
Anas Nashif
869ca055c5 sanitycheck: handle qemu crashes
Deal with qemu crashes and do not register pass status as the default.
In case we do not have a status, report unknown.

This now captures different states from qemu that were not captured
before.

Fixes #26679

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-07 16:04:32 -07:00
Erwan Gouriou
2339fa00f3 scripts: sanitylib.py: Add support for STLINK-V3 probe
Recent ST boards embed the new ST-Link probe V3.
It is advertised as "STLINK-V3", update sanitylmib to take it
into account.
In handle function, it is proposed to treat it separately as current
"STM32 STLink" as processing might differ in next future (hla_serial
deprecation).

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2020-07-07 16:41:09 -04:00
Øyvind Rønningstad
4813f469f0 sanitycheck: Improve handling of duplicate devices
Some devices have multiple serial ports, which show as duplicates in the
hardware map. This doesn't work well when using --generate-hardware-map
to regenerate an existing map. E.g. nrf5340pdk_nrf5340_cpuapp by default
prints to the 3rd of 3 devices. If it shows up on /dev/ttyACM0,
/dev/ttyACM1, and /dev/ttyACM2, printout will come on /dev/ttyACM2.
nrf9160dk_nrf9160 also provides 3 devices, but prints to the 1st.

This patch sorts the devices by the serial port and matches multiple
duplicate entries one-to-one to retain the ordering. This way, the
correct device can be given the platform name and the others can be kept
as "unknown" so that --device-testing understands correctly
1) that there is only one device (not 3)
2) where the serial output will come.

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2020-07-06 10:27:22 -04:00
Aastha Grover
8213a1509f sanitycheck: Testcases for report functions of TestSuite Class
test_reporting_testsuite.py: Adding testcases for reporting
functions of Class testSuite.
test_testsuite_class.py: Testcase for add_instances function
conftest.py: Added fixture for instances_fixture

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2020-07-03 12:20:35 -04:00
Aastha Grover
dcbd915759 sanitylib: Fix discard_report to check self.discards dict
Modifying discard_report function to fix the check for
self.discards dictionary in try block. self.discards
is None is dead code which didnot check if the dictionary was
empty. Changed it to if not self.discards.

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2020-07-03 12:20:35 -04:00
Anas Nashif
3cb1b437d2 ci: do not run sanitycheck on linter config changes
Do not run sanitycheck on linter configuration changes.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-03 07:58:31 -04:00
Anas Nashif
f04461e8d0 sanitycheck: fix --failed-only handling
- Report build errors as errors, not test failures
- Do not try and build/run tests with build failures
- Fix issue with empty reports when running --only-failed
- Report build errors in the detailed and target reports

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-03 07:25:49 -04:00
Michael Hope
bb8277962d scripts: bossac: automatically pull the flash offset from the config
BOSSAC uses a bootloader, so pull the flash address from DeviceTree or
the config and use if the version of BOSSAC supports offsets.

Signed-off-by: Michael Hope <mlhx@google.com>
2020-07-02 08:23:03 -04:00
Michael Hope
4ac195e0af scripts: bossac: add support for bossac 1.9.1
BOSSA 1.7 has built-in bootloader offset handling while 1.9.1
requires the user to supply the offset.  Add support for both by
sniffing the help output and warn the user if the flags needs adding.

Related to #22062

Signed-off-by: Michael Hope <mlhx@google.com>
2020-07-02 08:23:03 -04:00
Torsten Rasmussen
13642fe013 cmake: west: west flash dependencies in sync with CMake flash target
This commit creates a list of a phony targets for each runner, that is:
`west_flash_depends`, `west_debug_depends`, and so on.
Those targets has identical dependencies as CMake runner target.
flash, debug, debugserver, attach targets.

As example `ninja flash` correctly ensures dependencies are taken into
consideration before calling `west flash`.

Unfortunately, calling `west flash` directly would not re-run the flash
dependencies, cause `west flash` would only build the default CMake
target.

Now, `west flash` calls the phony `west_flash_depends` target, ensuring
all deps are up-to-date before flashing (unless --skip-rebuild is given)

The same is true for the other mentioned runners.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-07-01 07:58:10 -04:00