Commit Graph

1945 Commits

Author SHA1 Message Date
Carles Cufi
31bdad5299 west: Implement CMake helpers in scripts/west_commands
Move the existing CMake and build functionality from the west repository
to zephyr. The rationale behind this move is that it's very tightly
coupled with the Zephyr build system and is only used by the extension
commands implemented in the zephyr tree.
If additional extension commands in third-party repos want to use the
functionality they can add $ZEPHYR_BASE/scripts/west_commands to the
Python system path.

The implmentations in the west repo will be deprecated.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-05-03 21:55:13 +02:00
Sigvart M. Hovland
06365a5ed1 cmake: zephyr_modules: Add west executable argument
On some systems where you don't have access to `PATH` and you can't
set the `ENV{PATH}` variable. You need to be able to pass the path
to the west executable down to the python script so it is better
for it to be set explicitly than assuming that it exsists as a
part of the PATH/executables in the shell being called.

Signed-off-by: Sigvart M. Hovland <sigvart.hovland@nordicsemi.no>
2019-05-03 14:23:09 -04:00
Maureen Helm
d6d1e27a97 scripts: west_commands: Print error message when jlink exe not found
Adds a check in the jlink runner to look for the jlink executables and
print a more useful error message if they are not found.

Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
2019-05-02 22:50:55 -04:00
Luiz Augusto von Dentz
e0f3ab6bf2 Bluetooth: GATT: Add support for static services
This reintroduces support for static service in the form of a new API,
BT_GATT_SERVICE_DEFINE, and changes the internal services (GAP/GATT)
to be defined as const as they are never register/unregistered.

Internal service needed to be renamed in order to keep the same order
as before since the section elements are sorted by name.

The result is the following (make ram_report):

before:
      gatt.c                                    572     0.66%
        cf_cfg                                   32     0.04%
        db                                        8     0.01%
        db_hash                                  16     0.02%
        db_hash_work                             32     0.04%
        gap_attrs				180     0.21%
        gap_svc                                  12     0.01%
        gatt_attrs                              160     0.18%
        gatt_sc                                  80     0.09%
        gatt_svc                                 12     0.01%
        sc_ccc_cfg                               32     0.04%
        subscriptions                             8     0.01%

after:
      gatt.c                                    210     0.24%
        cf_cfg                                   32     0.04%
        db                                        8     0.01%
        db_hash                                  16     0.02%
        db_hash_work                             32     0.04%
        gatt_sc                                  80     0.09%
        last_static_handle                        2     0.00%
        sc_ccc_cfg                               32     0.04%
        subscriptions                             8     0.01%

Signed-off-by: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
2019-05-02 08:29:23 +03:00
Flavio Ceolin
bf878ced12 sanitycheck: Fix an error in scan_path exception
The function error expects only one parameter. The excpetion handler in
scan_path was calling this function with multiple parameters instead of
formatting the string.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2019-04-28 13:38:32 -04:00
Torsten Rasmussen
0dd3b421c4 west: zephyr_module: Improved error messaging during west failures
Fixes: #15664

This commit improve error messaging in case `west` list fails.
Previously any error messages and stack trace reported by `west` will
was thrown away by zephyr_module.py.

Now the error, as well as any stack traces, printed by `west` will be
re-printed to the user.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2019-04-27 12:38:32 -07:00
Marc Herbert
4a10eea0eb gen_kobject_list.py: better comments and --help. Zero code change.
- The script can but does not always generate five files, in fact the
  current build invokes it at least three times requesting different
  outputs every time.

- --kobj-types-output produces a code fragment; not a standalone/usable
  header file. It outputs enum constants for a single enum type and not
  several enum types.

- Some outputs include driver instances and others not: clarify which.

- There's an entire and great section in the documentation that took
  me ages to find because it's not referenced anywhere in the --help
  or code. Fixed.

- Highlight the massive duplication in the CMakeLists.txt to save
  déjà vu confusion and minimize future divergence.

- Other minor tweaks.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-04-26 03:21:59 -07:00
Carles Cufi
3a88dce9c5 scripts: west: Run pristine.cmake directly instead of the target
When making a build folder pristine until now we were running the
'pristine' build target. The issue with that is that ninja/make or
whatever build tool is being used might decide to re-run CMake itself if
some of the dependencies have changes. This might trigger an error that
is unfriendly and unnecessary, since the user is explicitly asking for
the build folder to be wiped before starting a fresh build.
To avoid this issue restor to running directly the CMake script that the
'pristine' build target itself uses, so as to make sure that the build
folder is wiped unconditionally regardless of changes made to the tree.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-04-23 13:24:41 -07:00
Anas Nashif
d18ec53575 sanitycheck: show handler in verbose mode
When running in verbose mode, show what handler is being used.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-20 14:14:47 -04:00
Anas Nashif
d6476eee3e sanitycheck: handle pid file not being present
Do not try to open non-existing pid files.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-20 14:14:47 -04:00
Anas Nashif
042d9b7d83 sanitycheck: fix reporting: build vs. run
If build fails, report a build failure instead of handler error.

Fixes #15272

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-20 14:14:47 -04:00
Anas Nashif
654ec598ac sanitycheck: add timestamps to messages
Using the new option --timestamps, any output from sanitycheck will have
a timestamp to help identify bottle necks and monitor execution time.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-20 14:14:47 -04:00
Anas Nashif
924987d72f ci: detect changes to sub-board definitions
When detecting changes to boards, make sure we test all board
configurations available in that board directory, not only the main one.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-20 14:14:29 -04:00
Patrik Flykt
83de530d5a scripts/coccinelle: Add script for counting identifier length
Add a simple Coccinelle script that counts identifier lengths and
prints out a warning if it is longer than 31 characters.

The script can be run with:
spatch -D report --very-quiet \
--include-headers --recursive-includes \
--cocci-file $ZEPHYR_BASE/scripts/coccinelle/identifier_length.cocci \
--dir $ZEPHYR_BASE \
kernel/

Where '--include-headers' and '--recursive-includes' can be omitted
if neede.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2019-04-18 12:29:07 -04:00
Andy Ross
aa2b8a1bc7 scripts/sanitycheck: Precompile test-time regular expressions
The Harness handlers for tests were parsing the realtime stream out of
qemu pipes by recompiling and executing every regex for every line (!)
of output from the simulator.  That's a significant CPU load, and it's
(1) in a separate thread not tracked by the JOBS limit and (2)
happening at the worst possible time and contending with the qemu
process for host CPU cycles that it needs to hit its (real world)
timer targets on time.

Compile them just once, please.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2019-04-18 12:21:13 -04:00
Ulf Magnusson
86b0c22dc4 kconfig: Avoid potential issue parsing generated_dts_board.conf
Splitting a string like 'foo="bar=baz"' on '=' will give ['foo', '"bar',
'baz"'] instead of the intended ['foo', '"bar=baz"']. split() with
maxsplit=1 to avoid potential issues.

Not seen in practice. Just some future safety.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-04-18 08:49:46 -04:00
Ulf Magnusson
aec74f692c kconfiglib/menuconfig: Various behavior/UI improvements
Update Kconfiglib and menuconfig to upstream revision 90c5573c19, to get
these commits in:

    Warn for unquoted argument to 'source', etc.

    Print a warning suggesting to add quotes for things like

      source foo/bar/Kconfig
      menu title
      prompt unquoted

    Example warning:

      Kconfig:32: warning: style: quotes recommended around
      'lib/Kconfig.debug' in 'source lib/Kconfig.debug'

    That quoteless syntax is supported for compatibility with old
    versions of the C tools. It only works for a single word.

    ==================================================================

    menuconfig: Include all parents in menu paths

    Previously, symbols not defined with 'menuconfig' with children
    weren't listed in the children's menu paths. It was deliberate, but
    it's probably an anti-feature in retrospect, because it can make it
    harder to find stuff by following the menu path.

    Don't try to be clever and just list all the parent nodes in the
    menu path.

    ==================================================================

    menuconfig: Fix display issue for optional-prompt menuconfigs

    _shown_nodes() needs to check whether invisible 'menuconfig' symbols
    with optional prompts have visible children, so that they can be
    shown outside show-all mode. Previously, only 'config' symbols were
    checked.

    ==================================================================

    menuconfig: Remember last saved/loaded path and improve
    _conf_changed

    Remember the last path that was manually saved/loaded instead of
    reverting back to standard_config_filename() (e.g. .config).

    Remember the path to the last saved minimal configuration separately
    as well.

    Also improve the _conf_changed behavior when loading a .config
    within the interface. Instead of always treating it as needing to be
    saved, check if it's outdated, like for the .config file loaded on
    startup.

    Also make the exit message ("No changes to save", etc.) always
    include the target .config file, which is helpful. Previously, only
    the save message did.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-04-18 08:46:15 -04:00
Carles Cufi
b7c75915e0 west: build: Add new pristine cmd-line and config option
Add a new command-line and build config option, `pristine`, that the
user can pass to `west build` or set in its configuration file(s) in
order to automatically trigger a pristine build on every build or
whenever west considers it required.

The option can take the following values:

- never: Never run the target
- always: Always run the pristine target before building
- auto: Run the pristine target when required

With `auto`, the pristine target will be run when running
west with an existing build folder containing a build system and:

- Selecting a different board from the one currently in the build system
- Selecting a different application from the one currently in the build
  system

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-04-17 10:02:46 -04:00
Carles Cufi
3c6584d6d5 west: build: Use officially documented CMake options
The -B option has always existed but was first officially documented in
CMake 3.13.0. In that same release the -S option was introduced,
replacing the old undocumented -H. Switch to using the officially
documented options.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-04-17 10:02:46 -04:00
Marc Herbert
0f7255c3b8 sanitycheck: order results.csv and discards.csv deterministically
One of the first things needed when comparing builds of tests across
different environments/systems is to make sure the same (sub)tests were
selected and run in the first place. For that purpose sort the output of
--testcase-report and --discard-report as they were in random order.

Actually make the entire class TestInstance sortable by adding a
standard __lt__() method comparing unique instance names; it could be
useful again.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-04-17 08:28:54 -04:00
Marc Herbert
3061c92aaf file2hex.py: new --gzip-mtime option that defaults to zero + test
This makes the output of file2hex.py deterministic by default while also
letting the user set the mtime in the gzip header manually if desired.

Use the option without any argument to restore the previous behavior
that sets the current (and obviously changing) "now" timestamp.

To test:  ./sanitycheck --tag gen_inc_file

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-04-17 08:27:40 -04:00
Anas Nashif
a94f2b0b12 release: update footprint data
Update footprint data for 1.14 release.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-15 20:31:52 -04:00
Ulf Magnusson
4f920df29a scripts: Remove checkconfig.py
This is an old script for finding references to undefined Kconfig
symbols that assumes Kconfiglib 1 (an older API).

There's now a different check for references to undefined symbols (see
commit 1d0834b35f ("checks: kconfig: Check for references to undefined
Kconfig symbols") in the ci-tools repo).

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-04-12 08:28:03 -04:00
Marti Bolivar
100a2bba6c scripts: update pyocd in requirements.txt
This includes a bugfix for a pyocd requirement on pyyaml
that could not be satisfied using officially supported
releases.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-04-10 17:52:15 -04:00
Andrew Boie
9e69c54060 sanitycheck: add more info during processing
After the testcase configs are built, there is a step to
filter all the test case information to determine the set
of tests to run.

As this step takes a nontrivial amount of time, add an
informational message about it.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-04-10 13:09:28 -04:00
Torsten Rasmussen
ad026c09b1 cmake: modules: using posix path for Kconfig generated file
Fixes: #15289

Kconfig requires posix style path when sourcing other files.
As abspath in python will use native host style, i.e. backslash '\' in
windows this will cause invalid paths in Kconfig generated file and
thus the file will never be loaded.

This commit uses PurePath to convert the path to posix style, ensuring
Kconfig can load the modules.

Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
2019-04-09 16:47:45 +02:00
Anas Nashif
f6a163fc5c ci: remove unused script merge_junit.py
This script is now part of ci-tools repo.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-07 10:26:27 -04:00
Anas Nashif
3ae52624ff license: cleanup: add SPDX Apache-2.0 license identifier
Update the files which contain no license information with the
'Apache-2.0' SPDX license identifier.  Many source files in the tree are
missing licensing information, which makes it harder for compliance
tools to determine the correct license.

By default all files without license information are under the default
license of Zephyr, which is Apache version 2.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-07 08:45:22 -04:00
Anas Nashif
8287f7c855 scripts: remove old xtools config files
Those files are outdated. We now have them as part of the sdk-ng git
repo.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-07 08:45:22 -04:00
Patrik Flykt
7c0a245d32 arch: Rename reserved function names
Rename reserved function names in arch/ subdirectory. The Python
script gen_priv_stacks.py was updated to follow the 'z_' prefix
naming.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2019-04-03 17:31:00 -04:00
Andrew Boie
f0835674a3 lib: os: add sys_mutex data type
For systems without userspace enabled, these work the same
as a k_mutex.

For systems with userspace, the sys_mutex may exist in user
memory. It is still tracked as a kernel object, but has an
underlying k_mutex that is looked up in the kernel object
table.

Future enhancements will optimize sys_mutex to not require
syscalls for uncontended sys_mutexes, using atomic ops
instead.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-04-03 13:47:45 -04:00
Andrew Boie
1dc6612d50 userspace: do not track net_context as a kobject
The socket APIs no longer deal with direct net context
pointers.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-04-03 13:47:45 -04:00
Andrew Boie
a00eff731c gen_kobject_list: allow object data to be C code
The data value in a kernel object structure is specific
to that type of object. Allow this to be a reference to
another C symbol or other compiled code by populating as
a string.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-04-03 13:47:45 -04:00
Andrew Boie
c235e167fa elf_helper: support for objects in user memory
Some forthcoming kernel object types like futexes need to
be tracked, but do not contain data that is private to
the kernel.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-04-03 13:47:45 -04:00
Andrew Boie
ce6210e479 gen_priv_stacks.py: fix kobject list
k_stack are orthogonal to thread stacks.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-04-03 13:47:45 -04:00
Anas Nashif
193fb971c2 ci: do not report coverage on pull requests
The code coverage reports on pull requests has been inconsistent and
confusing. Disable them for pull-requests.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-01 18:39:17 -04:00
Anas Nashif
366ed11bfa sanitycheck: deal with special charachters in xml
Some logging text and colors were not escaped correctly, make sure we
generate well formed XML reports.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-01 12:23:09 -04:00
Anas Nashif
f16e92c000 sanitycheck: count samples in reports
We have not been counting samples in reports. This change lists tests
associated with sample code which in many cases is just verifying output
from the sample and counts as 1 test.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-01 12:23:09 -04:00
Carles Cufi
766edd449d cmake: modules: Enclose name and path in quotes
Due to the fact that CMake only supports greedy regexes,
the ':' in a Windows path ('C:\...') was being matched leading
to an invalid split of the line. Quote both the name and the path
to avoid this issue.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-04-01 07:27:23 +08:00
Torsten Rasmussen
bd7569f272 cmake: Extracted Zephyr module processing into python script
Fixes: #14513

This commit move the functionality of extracting zephyr modules into
generated CMake and Kconfig include files from CMake into python.

This allows other tools, especially CI to re-use the zephyr module
functionality.

Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
2019-03-29 23:10:59 -04:00
Marc Herbert
195195a4fc file2hex.py: switch from gzip.compress() to GzipFile()
Zero functional change, this is pure refactoring and preparation for
using the mtime= parameter which the gzip.compress() shortcut does not
make available.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-03-29 08:12:53 -04:00
Patrik Flykt
2fb87b9dd2 scripts/coccinelle: Update unsigned suffix script
Update unsigned suffix script to properly match multiplication.
Fix provided by Himanshu Jha.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2019-03-28 17:15:58 -05:00
Patrik Flykt
caebf204c6 scripts/coccinelle: Update Coccinelle script for unsigned values
Update the script to detect and update more instances of unsigned
variable assignments when using all four simple rules of arithmetics.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2019-03-28 17:15:58 -05:00
Ulf Magnusson
a449c98db2 scripts: Fix risky uses of non-raw regex strings in Python scripts
Fixes pylint warnings like this one:

    doc/conf.py:325:0: W1401: Anomalous backslash in string: '\s'.
    String constant might be missing an r prefix.
    (anomalous-backslash-in-string)

The reason for this warning is that backslash escapes are interpreted in
non-raw (non-r-prefixed) strings. For example, '\a' and r'\a' are not
the same string (first one has a single ASCII bell character, second one
has two characters).

It just happens that there's no \s (or \., or \/) escape for example,
and '\s' turns into two characters (as needed for a regex). It's risky
to rely on stuff like that regexes though. Best to make them raw strings
unless they're super trivial.

Also note that '\s' and '\\s' turn into the same string.

Another tip: A literal ' can be put into a string with "blah'blah"
instead of 'blah\'blah'.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-28 14:41:32 -05:00
Ulf Magnusson
0e8e92cd87 scripts/dts: Fix bug in dead /memreserve/ code
The /memreserve/ code would crash if it ever ran, because 'name' isn't
defined (seems to be some copy-paste here). There are no /memreserve/s
in Zephyr though, so it works out.

'name' seems to be the node name. Not sure what to put for a
/memreserve/, but make it '<memreserve>' to make it stand out.

Fixes a pylint warning.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-28 11:11:13 -05:00
Ulf Magnusson
7ffd628dac scripts: elf_helper: Remove undefined var ref. in ArrayType.__repr__()
self.num_members doesn't exist. This commit just removes the reference,
because I didn't want to guess a proper fix.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-28 11:07:42 -05:00
Ulf Magnusson
425998bd79 scripts: elf_helper: Tidy up get_symbols() to eliminate pylint warning
Using a member variable in the dict comprehension was probably a typo
(can't see 'sym' referenced elsewhere). Use a local variable instead.

Made pylint spit out these warnings (which might be spurious though):

    scripts/elf_helper.py:535:24: E0203: Access to member 'sym' before
    its definition line 536 (access-member-before-definition)

    scripts/elf_helper.py:535:39: E0203: Access to member 'sym' before
    its definition line 536 (access-member-before-definition)

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-28 11:06:33 -05:00
Ulf Magnusson
d5b0bd14e3 scripts: Remove unused imports in all Python scripts
Discovered with pylint3.

Upstream open-amp PR: https://github.com/OpenAMP/open-amp/pull/168

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-28 11:06:20 -05:00
Ulf Magnusson
12ba9dfa52 scripts: Remove unused variables in all Python scripts
Discovered with pylint3.

Use the placeholder name '_' for unproblematic unused variables. It's
what I'm used to, and pylint knows not to flag it.

Python tip:

    for i in range(n):
        some_list.append(0)

can be replaced with

    some_list += n*[0]

Similarly, 3*'\t' gives '\t\t\t'.

(Relevant here because pylint flagged the loop index as unused.)

To do integer division in Python 3, use // instead of /.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-28 11:06:20 -05:00
Ulf Magnusson
399c04ce5a scripts/dts: Remove unused variables and imports
Discovered with pylint3.

Use the placeholder name '_' for unproblematic unused variables. It's
what I'm used to, and pylint knows not to flag it.

Also improve the naming a bit in devicetree.py. If a key/value is known
to be a specific thing (like a node), then it's helpful to call it that
instead of something generic like "value".

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-28 11:06:01 -05:00
Marc Herbert
682961a382 sanitycheck: document that --save-tests appends to existing file
Fix --help message. Also rename run_report() to save_tests() as it's
used only once by --save-tests and nowhere else. Maybe the code was
shared with some --other-report feature in the past but not any more.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-03-28 09:30:00 -04:00
Andrew Boie
cf91f8cf90 elf_helper: fix debug_die call
debug_die() is not implemented in this class, and indeed we
don't even have a reference to the DWARF DIE object.

This is a fatal error anyway, just raise an exception.

Fixes #14762

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-03-27 16:37:49 -05:00
Marc Herbert
6f011c95c4 sanitycheck: don't generate the top-level Makefile in random order
Also add a "generated by sanitycheck" header to indicate origin and a
warning about the dependency required to actually run the test.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-03-26 22:23:56 -04:00
Ulf Magnusson
e9c1d6d0ed scripts: gen_app_partition: Fix broken typo'd sys.exit()
os.exit() doesn't exist.

Also use the nifty sys.exit(msg) feature, which prints 'msg' to stderr
and exits with status 1.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-26 10:19:26 -05:00
Ulf Magnusson
ba312fe844 scripts: Remove unnecessary () around if/while conditions in Python
Not needed in Python. Detected by check C0325 in pylint3.

Also replace an

  if len(tag):

with just

  if tag:

Empty strings, byte strings, lists, etc., are falsy in Python.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-26 07:59:59 -05:00
Ulf Magnusson
d83141535c scripts: Remove accidental semicolons in Python scripts
Making a clean slate for some pylint CI tests. Only enabling relatively
uncontroversial stuff.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-26 07:57:52 -05:00
Ulf Magnusson
5c8fd84d12 scripts: Remove extra trailing newlines from Python scripts
Fixing all instances so that it can be flagged in a pylint CI check
later.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-26 07:53:08 -05:00
Ulf Magnusson
7da0053a0c scripts: gitlint: Fix broken module docstring
Needs to be at the beginning of the file. Fixes a pylint warning:

    scripts/process_gperf.py:26:-1: W0105: String statement has no
    effect (pointless-string-statement)

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-26 07:50:06 -05:00
Ulf Magnusson
1a27de0d31 scripts: gen_priv_stacks: Remove unused static kernel object stuff
Might've been a copy-paste from scripts/gen_kobject_list.py.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-26 07:47:36 -05:00
Ulf Magnusson
605423f9b1 scripts: process_gperf: Fix broken module docstring
Needs to be at the beginning of the file. Fixes a pylint warning:

    scripts/process_gperf.py:26:-1: W0105: String statement has no
    effect (pointless-string-statement)

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-26 07:46:11 -05:00
Marc Herbert
932a33ad75 sanitycheck: fix, expand and clarify test case selection --help
- Fix wrong --test help message
- Provide more examples to clarify naming hierarchy
- Document that --sub-test runs its entire --test
- Point out that save/load options use their own format
- Document that --list-tests is flattened

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-03-18 22:07:25 -07:00
Andrew Boie
7ea211256e userspace: properly namespace handler functions
Now prefixed with z_hdlr_ instead of just hdlr_.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-03-18 09:23:11 -07:00
Ulf Magnusson
958e2d3229 scripts/dts: Refactor Flash code to be less twisty
- Remove DTFlash.extract(), which was just dispatching to either
   _extract_flash() or _extract_code_partition() depending on which
   magic string was passed in. Call them directly instead.

 - Fold constant and globally available parameter values into functions

 - Remove DTFlash._flash_node. It's easy to derive wherever it's needed,
   and it makes it clearer where it comes from (and means functions can
   be called in any order).

 - Remove DTFlash._flash_base_address, which is unused

 - Remove various unused parameters to functions

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-15 11:09:49 -05:00
Ulf Magnusson
2a4d8b07aa scripts/dts: Remove weird def_label.split('/') code
def_label is the name used for macros and .conf file values. It can
never have a slash in it.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-15 11:09:49 -05:00
Ulf Magnusson
f95a230b82 scripts/dts: Clean up/fix extract/interrupts.py a bit
- Fix broken code that was meant to turn the
   'interrupts'/'interrupts-extended' value into a list if it wasn't.
   list(123) will error out instead of creating [123].

 - Remove weird .split('/') on macro name

 - Rename 'props' to 'vals'. It's the value of a single property.

 - Get rid of a bare 'except:'

 - Rename l_fqn to full_name. Accidentally stumbled upon 'fqn' probably
   standing for "fully qualified name", but it's not obvious.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-15 11:09:49 -05:00
Ulf Magnusson
3b64e71221 scripts/dts: Simplify extract/interrupts.py and add some helpers
- Have get_parent_path() return None for the root ('/'). This is handy
   when looping over path components.

 - Move _find_parent_irq_node() out of the class, call it
   parent_irq_node(), and use get_parent_path() in it.

 - Add a global err() function for reporting errors. Use it if a node
   unexpectedly has no interrupt-parent.

   Previously, this would give a Python error instead.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-15 11:09:49 -05:00
Ulf Magnusson
a3d3aa58b8 scripts/dts: Remove redundant empty constructors
It's not obligatory for Python classes to have an __init__().

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-15 11:09:49 -05:00
Ulf Magnusson
0a3f00ad7c scripts/dts: Rename get_node_label() to node_label()
Just returns a value, with no side effects.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-15 11:09:49 -05:00
Kumar Gala
4da0f8b796 linker: Remove unused (OPTIONAL) from linker scripts
(OPTIONAL) was a vestiage from the initial import of the Zephyr code
base and we dont utilize it with the GNU linker.  Additionally, the way
(OPTIONAL) gets defined to nothing creates a linker script that lld
(from llvm) doesn't like.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-03-15 06:42:02 -05:00
David B. Kinder
6000a6205a doc: prepare for improving doc API linking
Linking to API material requires knowing the pecularities of how
doxygen, sphinx, and breathe work. In an attempt to hide some of this
we're preparing the current docs to allow use of configuration defaults
that will let us more simply use a default role that will hunt for a
reference target in the various domains that are available by using a
default "role" of "all".  This will let us use the simple notation
`functionname` or `typename` without fully specifying the reference as
:c:func:`functionname`.

This patch cleans up exising docs that were (incorrectly) using single
backtics where double backtics should have been used, and also found
some typos (such as a space between the role name and the reference,
such as :file: `filename`, and a missing colon such as
c:func:`functionname`)

This is a start to address issue #14313

Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
2019-03-15 05:47:19 +01:00
Michael Scott
4278bf5673 scripts/dts: Fix for PyYAML 5.1 Loader changes
PyYAML 5.1 was just released and it doesn't support !include as
previous versions do.  This breaks our DTS bindings parsing.

Let's fix our extract_dts_include.py script to work with both
3.13 and 5.1.

Also, update the pyyaml requirement to >=3.13 to be sure we're
compatible.

Fixes: https://github.com/zephyrproject-rtos/zephyr/issues/14496

Signed-off-by: Michael Scott <mike@foundries.io>
2019-03-13 23:21:42 -05:00
Kumar Gala
1230ae1e63 scripts/sanitycheck: Add 'vectors' section to whitelist
Add 'vectors' to the whitelisted list of linker sections.  This is
needed by the openisa_rv32m1 SoC.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-03-13 17:02:05 -05:00
Daniel Leung
212ec9a29a linker: sort app shared mem partition by alignment
If CONFIG_MPU_REQUIRES_POWER_OF_TWO_ALIGNMENT is enabled,
the app shared memory partition may cause waste of memory
due to the need for padding.

For example, tests/subsys/jwt and board mps2_an385:

  z_test_mem_partition: addr 0x20000000, size 52
  z_libc_partition    : addr 0x20000040, size 4
  k_mbedtls_partition : addr 0x20008000, size 32736

    ending at 0x2000ffff, taking up 65536 bytes

With power-of-two size and alignment requirement,
k_mbedtls_partition takes up 32KB memory and needs to be
aligned on 32KB boundary. If the above partitions are
ordered as shown, there needs to be a lot of padding
after z_libc_partition before k_mbedtls_partition can
start. In order to minimize padding, these partitions
need to be sort by size in descending order.

After the changes here,	the partitions are:

  k_mbedtls_partition : addr 0x20000000, size 32736
  z_test_mem_partition: addr 0x20008000, size 52
  z_libc_partition    : addr 0x20008040, size 4

    ending at 0x2000805f, taking up 32864 bytes

With the above example, sorting results in a saving
of 32672 bytes of saving.

Fixes #14121

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2019-03-13 18:26:51 +01:00
Erwan Gouriou
7f84787c13 scripts: fix path issue in push-hook script
series-puh-hook.sh script was pushed with local path
instead of ${ZEPHYR_BASE}.
Fix this.

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2019-03-13 11:59:38 -05:00
Marc Herbert
0c465bbec1 sanitycheck: make --subtest and --test mutually exclusive
They're effectively mutually exclusive already because
options.sub_test in main() immediately discards any --test
argument(s). This commit preempts user confusion thanks to this new
message:

 sanitycheck: error: argument --sub-test: not allowed with
    argument -s/--test

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-03-13 01:44:52 +01:00
Carles Cufi
c5daabe190 west_commands: pyocd: Sector erase by default
Due to issues with the implementation of the default auto-erase in
pyocd, sometimes the chip is mass-erased even when not intended. To
avoid this issue, default to forcing sector erasing unless mass erasure
is explicitly requested by the user with additional flash options.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-03-12 16:52:14 +01:00
Andrew Boie
c78c5e6936 userspace: add additional script documentation
We have several scripts used by the build system related
to generating code for system calls, privileged mode stacks,
kernel object metadata, and application shared memory
partitions. Add some overview documentation for each.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-03-12 08:37:58 +01:00
Marc Herbert
edf17591dd sanitycheck: group case selection options in the help message
$ sanitycheck -h # is re-ordered like this:

 < ... all other options ... >

 -C, --coverage        Generate coverage reports. Implies
                       --enable_coverage

 --coverage-platform   COVERAGE_PLATFORM
                       Plarforms to run coverage reports on. This
                       option may be used multiple times.

Test case selection:

 -f, --only-failed     Run only those tests that failed the previous
                       sanity check invocation.

 -s TEST, --test TEST  Run only the specified test cases. These are
                       named by <path to test project relative to
                       --testcase-root>/<testcase.yaml section name>

 --sub-test SUB_TEST   Run only the specified sub-test cases and its
                       parent. These are named by test case name
                       appended by test function, i.e.
                       kernel.mutex.mutex_lock_unlock.
 --list-tests          list all tests.
 -F FILENAME, --load-tests FILENAME
                       Load list of tests to be run from file.
 -E FILENAME, --save-tests FILENAME
                       Save list of tests to be run to file.
 -T TESTCASE_ROOT, --testcase-root TESTCASE_ROOT
                       Base directory to recursively search for test
                       cases. All testcase.yaml files under here will
                       be processed. May be called multiple times.
                       Defaults to the 'samples' and 'tests' directories
                       in the Zephyr tree.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-03-12 08:32:49 +01:00
Patrik Flykt
4344e27c26 all: Update reserved function names
Update reserved function names starting with one underscore, replacing
them as follows:
   '_k_' with 'z_'
   '_K_' with 'Z_'
   '_handler_' with 'z_handl_'
   '_Cstart' with 'z_cstart'
   '_Swap' with 'z_swap'

This renaming is done on both global and those static function names
in kernel/include and include/. Other static function names in kernel/
are renamed by removing the leading underscore. Other function names
not starting with any prefix listed above are renamed starting with
a 'z_' or 'Z_' prefix.

Function names starting with two or three leading underscores are not
automatcally renamed since these names will collide with the variants
with two or three leading underscores.

Various generator scripts have also been updated as well as perf,
linker and usb files. These are
   drivers/serial/uart_handlers.c
   include/linker/kobject-text.ld
   kernel/include/syscall_handler.h
   scripts/gen_kobject_list.py
   scripts/gen_syscall_header.py

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2019-03-11 13:48:42 -04:00
Daniel Leung
c8066c55fc linker: remove references to _k_task_list/_k_event_list
Commit 45a7e5d076 removed scripts/sysgen
and _k_task_list/_k_event_list sections are no longer being used.
So remove them.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2019-03-09 16:01:23 -05:00
Daniel Leung
bd40fc2c90 linker: remove references to no longer exist _k_memory_pool
Commit 73cb9586ce renamed linker
section from _k_memory_pool to _k_mem_pool, but the references
in linker scripts are still there. So remove them now.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2019-03-09 16:01:23 -05:00
Erwan Gouriou
831fbed3a7 scripts: Add hook calllable series-push-hook script
Running checkpatch in pre-commit hook could be a constraint
for day to day work. Though, running it before pushing to
zephyr repo is highly recommended.
Add a  series push script callable from pre-push hook.
Document how to use it in contribute section.

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2019-03-08 16:43:06 -05:00
Carles Cufi
b710177a27 west: commands: build: Specify source dir without a flag
In order to simplify the usage of `west build`, take a positional
argument with the source directory instead of requiring the `-s,
--source-dir` flag. This makes it easier and quicker to invoke west when
building, as well as being consistent with CMake.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-03-08 21:19:42 +01:00
Sebastian Bøe
f96c9bc75a kconfig: Have ninja Re-run CMake when Kconfig sources change
Users often get confused when they change Kconfig sources and then
rebuild only to discover that nothing happens. To fix this we add a
dependency between re-running cmake, and all Kconfig sources, similair
to how touching CMakeLists.txt files cause CMake to re-run.

This fixes #5634

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2019-03-07 10:40:00 +01:00
Marc Herbert
f78288be20 gen_kobject_list.py: OrderedDict for < 3.6 determinism
Dictionaries are iterated in a random order by Python 3.5 and before.
This could have caused "Unstable" CI in PR #13921 and maybe others.
Anyway we want builds to be determimistic by default. Explicit
randomness can be added for better coverage but not by default.

1. When running "make kobj_types_h_target" repeatedly one can observe
that the following .h files keep changing in
build/zephyr/include/generated/:

- kobj-types-enum.h
- otype-to-str.h
- otype-to-size.h

Switching kobjects to OrderedDict makes these 3 .h files deterministic.

2. When running this test repeatedly with CONFIG_USERSPACE=y:

  rm build/zephyr/*.gperf && make -C build obj_list

... the dict used for --gperf-output seems to be deterministic, probably
because its keys are all integers (memory addresses). However we can't
take that for granted with Python < 3.6 so out of caution also switch
the output of find_objects() in elf_helper.py to a sorted OrderedDict.

PS: I would normally prefer official Python documentation to
StackOverflow however this one is a good summary and has all the
multiple pointers to the... official Python documentation.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-03-06 20:58:58 -05:00
Marc Herbert
73cb0bf306 extract_dts_includes.py: deterministic flash table with Python < 3.6
To reproduce the issue this addresses, run "cmake -B96b_nitrogen"
twice with Python 3.5. Observe the order of the flash partition table
changing in:
"build{1,2}/zephyr/include/generated/generated_dts_board.conf" and
"generated_dts_board_unfixed.h"

Dictionaries are iterated in a random order by Python 3.5 and before.
This could have caused "Unstable" CI in PR #13921 and maybe others.
Anyway we want builds to be determimistic by default. Explicit
randomness can be added for better coverage but not by default.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-03-06 20:58:58 -05:00
Ulf Magnusson
ac059db362 kconfiglib: Give more helpful hints for missing files
Sigvart pointed out that the error message for missing Kconfig files
(which rambles on a lot about $srctree) can easily kick people off in
the wrong direction.

Update Kconfiglib to upstream revision 9f26eb3ae3 to add the following
commit, which makes the error message point to stuff that's more likely
to be the problem:

    Give more helpful error messages when files are missing

    The current error message talks a lot about $srctree, but $srctree
    is seldom the culprit in practice. More common is 'source
    "$(SOME_ENV_VAR)/foo"`, where SOME_ENV_VAR hasn't been set.

    Include the complete 'source ...' line for missing Kconfig files,
    and mention unset environment variables as a hint. Only mention
    $srctree briefly.

    Also shorten the message when a .config can't be a found a bit. This
    message would usually only be seen when working directly with the
    library.

Example error:

    Kconfig:7: '/Kconfig' not found (in 'source
    "$(SOME_ENV_VAR)/Kconfig"'). Check that environment variables are
    set correctly (e.g. $srctree, which is unset or blank). Also note
    that unset environment variables expand to the empty string.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-06 12:07:50 +01:00
Andrew Boie
f084c38d44 userspace: properly namespace linker vars
App shared memory defines a bunch of symbols in the
linker script. Namespace them properly as private
zephyr variables.

The variables which indicate the bounds of the entire
partition now end with "_part_start", "_part_size",
and "_part_end" to make them easy for scripts to
distinguish them from other generated symbols for
data/bss sizes.

Finally, the bss size is not rounded up, this was
causing unnecessary memory to be zeroed.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-03-05 10:02:27 -08:00
Anas Nashif
f4d8eccb70 sanitycheck: fixed handling of retries
When re-running failed tests, do not go through any filtering, just load
the failed tests directly and execute them. The filtering was done based
on default platforms and any tests that were failing on non-default
platforms were ignored.

Fixes #13956

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-03-04 14:08:12 -05:00
Anas Nashif
c9d1bb90d8 ci: add option to specify custom range
This is useful when running the script locally on a set of commits to
reproduce how things are run in CI.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-03-04 14:08:12 -05:00
Anas Nashif
5923760dd5 sanitycheck: handle serial exception
Do not throw excpetion when there is no serial output, just timeout.

Fixes #13541

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-03-04 08:14:29 -05:00
Marc Herbert
6b4cda9b25 run_ci.sh: git log what [base] version is being tested
Zephyr Continuous Integration made the choice of rebasing pull requests
on its fast moving master branch before testing them. Unfortunately the
Shippable interface assumes the other choice[*] and highlights the SHA1
of the pull request _before_ rebasing which is very misleading.  For now
the only way to realize this disconnect is to scrutinize the build logs
and/or this source code (or tribal knowledge).

To mitigate and reduce a bit this confusion, run a short:
  git log -n 5 --oneline --decorate
before and after rebase.

Making it worse, Shippable sub-runs are not pegged to a common base
version. For instance the reason why shippable sub-run 35578.1 passed
while all other 35578.x sub runs failed is very likely because PR #13803
was merged to master in the mean time. This commit would have been able
to prove that.

[*] In an ideal world with infinite time and resources, Continuous
Integration would test pull requests BOTH alone and combined with the
moving tip.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-03-01 09:39:31 +01:00
Marc Herbert
f7e995c075 run_ci.sh: run pytest-3 if found
On (at least) Fedora "pytest" is python2 and python2 only.

Also update remote name in sample invocation from "upstream" to "origin"
because that's how west sets things up by default (and this script
requires west)

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-03-01 09:39:31 +01:00
Paul Sokolovsky
c2a70a2710 scripts: size_report: Fix output of header break line
Was apparently an artifact of Python2 to Python3 conversion. Led to
printing of literal '='*110i instead of a line of ='s.

Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
2019-02-28 09:23:16 -08:00
Marc Herbert
29c19c1b0f filter-known-issues.py: clarify what "new" means
When building incrementally, filter-known-issues.py reports a varying
and totally different set of "new" issues than when building from
scratch. Warnings for unrelated upstream code disappearing and
re-appearing are especially confusing. Expand the messages a bit to
clarify.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-02-28 18:18:30 +01:00
Ulf Magnusson
d3bf4b432c scripts/dts: Remove dead 'aliases' check in insert_defs()
insert_defs() is the only function that adds keys to 'defs', and they
always get added with an 'aliases' key.

This function could get super simple if the duplicate-removal code could
be removed. It only seems to be needed to prevent duplicates for some
(unfortunately tricky) flash-related stuff now.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-27 10:02:35 -06:00
Ulf Magnusson
25ff4b72a9 scripts/dts: Remove global 'structs' variable
Unused.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-27 10:02:35 -06:00
Ulf Magnusson
59a0c4331d scripts/dts: Fix binding_compats/bindings_compat naming mismatch
'binding_compats' makes the most sense, so use that everywhere.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-27 10:02:35 -06:00
Ulf Magnusson
5999f10ba2 scripts/dts: Get rid of deepcopy() and 'except:' in translate_addr()
- No deep copying is needed. The deepcopy() is just a hack to work
   around that the value might not be a list. That deserves a comment at
   least.

 - A catch-all try/except will hide stuff like misspelled variables as
   well. It's usually a bad idea.

Rewrite things to be more explicit, and add some comments.

reg.py was indirectly using the deepcopy() imported in global.py,
because it does 'from global import *'. Have it import deepcopy() itself
instead. reg.py is the only remaining user of deepcopy().

reg.py relying on the deepcopy() import in global.py was super hard to
discover, due to another catch-all try/except around the deepcopy()
call.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-27 10:02:35 -06:00
Ulf Magnusson
9d4c2921ee scripts/dts: Remove redundant 'children' check in create_chosen()
All nodes have a 'children' key.

Also simplify a bit. The loop is copying a dictionary verbatim.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-27 10:02:35 -06:00
Ulf Magnusson
5edeec3dc4 scripts/dts: Simplify and document get_compat()
Get rid of the try/except and the find_parent_prop() call. Also add some
documentation.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-27 10:02:35 -06:00
Ulf Magnusson
e0f55f17e8 scripts/dts: Fix get_parent_path() for top-level nodes
'/foo' was turned into ''. Turn it into '/' instead. That's how the root
is represented in 'reduced' too.

Also remove some code from get_addr_size_cells() that was only there to
work around the bug.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-27 10:02:35 -06:00
Ulf Magnusson
bc73ba6fc3 scripts/dts: Remove get_node_compats()
Unused.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-27 10:02:35 -06:00
Ulf Magnusson
392587b2a5 scripts/dts: Remove redundant 'children' check in create_aliases()
All nodes have a 'children' key.

Also simplify and improve naming a bit.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-27 10:02:35 -06:00
Ulf Magnusson
246a15c4db scripts/dts: Never ignore properties in generate_node_defines()
If a binding specifies 'generate: define' for 'interrupt-names' (like
some bindings do), then that ought to generate #defines for it, but the
scripts/dts code currently hardcodes 'interrupt-names' to be ignored
(along with some other properties).

Maybe the 'generate: define' in those bindings is a mistake, but the
code still ought to respect it. That also gets rid of some mystery code.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-27 10:02:35 -06:00
Andrew Boie
cfde940d6d x86: relocate some arch-specific scripts
These scripts are specific to the x86 port.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-02-25 13:15:37 -08:00
Andrew Boie
4ce652e4b2 userspace: remove APP_SHARED_MEM Kconfig
This is an integral part of userspace and cannot be used
on its own. Fold into the main userspace configuration.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-02-23 07:43:55 -05:00
Ulf Magnusson
79906fc481 scripts/dts: Document the generate_*_defines() functions
Add some short doc comments at the beginning so that people can quickly
get an idea of what they're about.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-22 12:44:16 -06:00
Ulf Magnusson
e04139f506 scripts/dts: Rename extract_property() to generate_prop_defines()
Consistent with generate_defines() and generate_node_defines().

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-22 12:44:16 -06:00
Ulf Magnusson
f5b17d4138 scripts/dts: Call /foo/bar a "path" instead of an "address"
It's confusing that "address" is often used within the same function to
refer to both node paths and e.g. address cells.

Make things easier to understand by calling /foo/bar a path instead.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-22 12:44:16 -06:00
Ulf Magnusson
5b791d1eea scripts/dts: Generate node-level #defines outside extract_property()
extract_property() is meant to generate #defines for a single property,
like 'foo = <1 2 3>'. Currently, it also generates node-level #defines
related to parent buses.

That makes the intent of the code hard to understand, and also means
that identical node-level #defines get redundantly added multiple times
(once per property).

Generate the node-level bus #defines before processing properties, in
generate_node_defines(). Use a new generate_bus_defines() helper.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-22 12:44:16 -06:00
Ulf Magnusson
edc1f6af7a scripts/dts: Move flash partition handling out of property loop
Due to the way the code was structured,
flash.extract_partition(node_path) was called multiple times for the
same 'node_path'.

That must've been a mistake to begin with (but was hard to spot before
the code was cleaned up). Move the generation of per-node #defines out
of the property loop.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-22 12:44:16 -06:00
Carles Cufi
c2d5e7b01c kconfig: Don't load env var if in doc mode
Do not load the GENERATED_DTS_BOARD_CONF if in doc mode, since it will
not defined as it doesn't apply. No need to defined it to a dummy value.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-02-22 10:14:21 +01:00
Andrew Boie
42d1481fb0 gen_mmu_x86: fix overlapping region detection
Fixes: #13515

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-02-21 09:49:19 -08:00
Ulf Magnusson
858701f67a scripts/dts: Remove redundant compat check in generate_defines()
generate_node_defines() immediately returns if the node's 'compatible'
is not in the binding.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-20 12:17:31 -06:00
Ulf Magnusson
75c6d2cd02 scripts/dts: Simplify generate_node_defines()
Merge stuff that no longer needs to be separate. Also improve naming.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-20 12:17:31 -06:00
Ulf Magnusson
797eaa78f6 scripts/dts: Remove redundant extract_property() parameters
Derive 'node_compat' and 'prop_val' (the dictionary for the property
from the YAML binding) inside extract_property().

That gives it just two parameters and makes it clearer that it's just
generating #define's for a single device tree property. 'prop_val' was
only used to look up prop_val['type'].

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-20 12:17:31 -06:00
Ulf Magnusson
e539479d4f scripts/dts: Move prop_names() call into extract_property()
Moves it closer to where it's used, and will allow other simplifications
in generate_node_defines().

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-20 12:17:31 -06:00
Ulf Magnusson
7bb57fe71f scripts/dts: Remove confusing deepcopy() in generate_node_defines()
After lots of reverse-engineering, I understand why deepcopy() is used:

 1) Various functions defined in scripts/dts/extract/* modify the list
    of names in-place

 2) A plain list.copy() won't work when the *-names property happens to
    have a single name in it, because devicetree.py gives a string
    instead of a list in that case

Using deepcopy() to solve (2) is very confusing (especially with no
comments), because no deep copying is actually needed.

Get rid of deepcopy(), add a helper function for fetching the names, and
some comments.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-20 12:17:31 -06:00
Ulf Magnusson
50ab1d8e5f scripts/dts: Clean up create_reduced()
- Remove dead "if 'props' in nodes" check

 - Rename the 'nodes' parameter to 'node'. It represents a single node.

 - Use a defaultdict(int) for 'last_used_id' (makes unset keys give 0)

 - Use a global for 'last_used_id'

 - Move some related code so it appears together

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-20 12:17:31 -06:00
Ulf Magnusson
8c3aabb2c2 scripts/dts: Remove find_node_by_path()
Unused.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-20 12:17:31 -06:00
Ulf Magnusson
07abb05bd9 kconfiglib: Clarify kconfig_filenames doc re. absolute paths
Update Kconfiglib (and menuconfig, just to sync) to upstream revision
99a7af769352b, to add the commit below, for a doc issue reported by
Sebastian Bøe:

  Document that kconfig_filenames keeps absolute paths as-is

  Came up in https://github.com/ulfalizer/Kconfiglib/issues/67.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-20 11:22:17 -06:00
Paul Sokolovsky
26e562c0bd scripts: gitlint: Consistent case for Signed-off-by
The commit header as added by "git commit -s" is "Signed-off-by:",
and not any other casing. While it perhaps case-insensitive, use
canonical casing to not confuse (attentive) users.

Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
2019-02-19 12:12:42 -05:00
Ulf Magnusson
a8c851771c scripts/dts: Remove block in extract_property()
Turn

  if a:
    if b:
      ...

into

  if a and b:
    ...

Simplify some error messages with .format() as well, and get rid of some
redundant str()s. '{0} {1} {1}'.format('foo', 'bar') gives
'foo bar bar'.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-19 07:44:44 -06:00
Ulf Magnusson
af3580c314 scripts/dts: Clean up generate_node_defines() a bit
- Reduce the indentation and remove the awkward line breaks. This is
   possible now that there's no recursion.

 - Be consistent with quotes

 - Move some initialization closer to where it's used

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-19 07:44:44 -06:00
Ulf Magnusson
44467ef8dc scripts/dts: Remove recursion from generate_node_defines()
The recursion was used to implement 'properties:' within 'properties:'
in binding files, which seems to be a dead leftover (and undocumented).

Removing it gets rid of code and makes things more transparent.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-19 07:44:44 -06:00
Ulf Magnusson
b80905fd8e scripts/dts: Inline filter_list in generate_node_defines()
Makes it clear that it's a static list, and moves the values closer to
where they're used.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-19 07:44:44 -06:00
Ulf Magnusson
c20598c84f scripts/dts: Clarify generate_defines() a bit
Use more descriptive naming, get rid of unused 'v' variable, and an
unnecessary None check (None won't appear in get_binding_compats()).

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-19 07:44:44 -06:00
Ulf Magnusson
3c1d9e893b scripts/dts: Remove unnecessary 'reduced' parameter
...in generate_node_defines(). 'reduced' is available as a global.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-19 07:44:44 -06:00
Ulf Magnusson
90c73e3e5a scripts/dts: Rename extract_node_include_info()
...to generate_node_defines().

More explicit. "extract node include info" can be read in many ways:

 - Extract a node's "include info"

 - Extract node and include info

 - etc.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-19 07:44:44 -06:00
Ulf Magnusson
0e40c9615b scripts/dts: Move DT_CHOSEN_* generation into generate_defines()
That way it consistently generates all #define's.

Add some related clarifying comments to main() too.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-19 07:44:44 -06:00
Ulf Magnusson
cbf9afccbe scripts/dts: Rename generate_node_definitions() to generate_defines()
As in #define's. Easier to guess (to me anyway).

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-19 07:44:44 -06:00
Ulf Magnusson
1ca73e33a9 scripts/dts: Use re.fullmatch() instead of re.match(... + '$', ...)
Simpler, and makes it clearer that the regex'ing is deliberate.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-19 07:44:44 -06:00
Ulf Magnusson
95385744f9 scripts/dts: Fix indentation in extract_node_include_info()
The for-loop body was indented by 8 spaces instead of 4.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-19 07:44:44 -06:00
Maureen Helm
6de4578dc6 scripts: Update pyocd version to 0.16.1
Update the pyocd version to pick up a fix for the command 'pyocd erase',
which was broken in 0.15.0 and fixed in 0.16.0. Although we don't have
any in-tree uses of this command in zephyr, there will soon be in-tree
uses in mcuboot when the deprecated 'pyocd-flashtool' and 'pyocd-tool'
commands are replaced with the new unified pyocd commands.

Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
2019-02-18 22:34:25 -05:00
Anas Nashif
f9e73c90f2 sanitycheck: do no overide OVERLAY_CONFIG
We have been overriding OVERLAY_CONFIG coming from tests. Concat all
files and create one variable that we pass to kconfig as list.

Fixes #13320

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-02-18 10:49:08 -05:00
Marc Herbert
d5b2834f58 scripts/*syscalls.py: sort os.walk() for a more deterministic build
Found by "disordered --shuffle-dirents=yes".

Sorting os.walk() in scripts/subfolder_list.py removes the randomness in
the following files:

  build/zephyr/misc/generated/syscalls_subdirs.txt
  build/zephyr/CMakeFiles/syscall_list_h_target.dir/ or build.ninja

Sorting os.walk() in scripts/parse_syscalls.py removes the randomness
in:

  build/zephyr/misc/generated/syscalls.json
  build/zephyr/include/generated/syscall_dispatch.c
  build/zephyr/include/generated/syscall_list.h

Note my (limited so far) testing did *not* observe any randomness in any
object file that this would address; the main purpose here is to remove
a very large amount of noise in diffoscope.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-02-17 07:32:05 -05:00
Kumar Gala
e6393dd22e scripts/sanitycheck: Add 'nocache' section to whitelist
Add 'nocache' to the whitelisted list of linker sections.

Fixes #13449

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-16 22:14:34 -05:00
Marti Bolivar
4a0f1f2817 west: sign.py: usability fixes
Fix some usability issues with this command.

- add help making it clear that either imgtool must be installed, or
  the path to imgtool.py must be provided using --tool-path.

- in case people don't read that, print a helpful message if imgtool
  is not installed and --tool-path is not provided.

- in case the build directory is not properly set up for an MCUboot
  chain-loaded image, make the BuildConfiguration inspection more
  robust, printing out errors using check_force() when values are
  missing.

- mark the --tool option required to print usage and avoid a
  RuntimeError if it is not provided.

- make sure we search for the default build directory before checking
  for its existence, in case it was not provided

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-02-15 21:07:10 +01:00
Marti Bolivar
f13fa537d7 west: fix issue with sign.py
The arguments have to be saved in self.args for check_force's error
path to work.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-02-15 21:07:10 +01:00
Tomasz Gorochowik
8adf39aae3 scripts: gen_kobject_list: Fix error log call
A trivial bug that caused the following error when this situation
actually happened:

  Too many thread objects (21)
  Traceback (most recent call last):
    File "/zephyr/scripts/gen_kobject_list.py", line 307, in <module>
      main()
    File "/zephyr/scripts/gen_kobject_list.py", line 281, in main
      -(-thread_counter // 8))
  TypeError: write() takes exactly one argument (2 given)

With this commit, the detected error is printed correctly, e.g:

  Too many thread objects (21)
  Increase CONFIG_MAX_THREAD_BYTES to 3

Signed-off-by: Tomasz Gorochowik <tgorochowik@antmicro.com>
2019-02-15 10:19:52 -08:00
Andrew Boie
eea0f93a83 x86: generate user mode page tables
If kernel page table isolation is enabled, we generate a second
set of page tables. These tables, except for the shared page, have
all non-user pages marked as non-present.

The MMU generation script has been refactored:

- Debugging output has been make significantly simpler and less
  verbose

- Useless globals removed or adjusted

- MMU region list is validated as it is read

- Some tuples unpacked into individual variables to make the
  code easier to read.

- Useless command line option for output binary endian-ness
  remobved

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-02-14 12:46:36 -05:00
Ulf Magnusson
732fa65956 scripts/dts: Rename dict_merge() to merge_properties()
It's special-cased for merging bindings and not a general dictionary
merging function.

Also simplify the documentation a bit.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-14 10:49:44 -06:00
Ulf Magnusson
14de997f25 scripts/dts: dict_merge(): Rename dct/merge_dct to to_dict/from_dict
Makes it easier to remember what's being merged into what. "merge_dct"
is ambiguous.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-14 10:49:44 -06:00
Ulf Magnusson
298b4431b9 scripts/dts: Do title/version/description special-casing in dict_merge()
Simpler to do it there than in merge_included_bindings().

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-14 10:49:44 -06:00
Ulf Magnusson
f0eeb11fc0 scripts/dts: Add helper for reporting !include-related errors
Makes the message end up on stderr along with the exception, and removes
some code.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-14 10:49:44 -06:00
Ulf Magnusson
35b3d440c1 scripts/dts: Replace the Bindings class with plain functions
The Bindings class was only used to implement '!include foo.yaml'
(easier to see after some things were moved out of it). Use plain
functions instead, which might be a bit more transparent and gives
simpler code.

Also remove the recursive '!include' detection for now, which is broken
in that the same .yaml being included twice will always trigger it, even
for non-circular cases.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-14 10:49:44 -06:00
Ulf Magnusson
4a99dbcf9e scripts/dts: Simplify and remove dead code in Bindings._extract_file()
As written, this function could take e.g. '!include bar', find 'foo-bar'
(due to the endswith()), and include it. This is undocumented and
doesn't seem to be used, so I'm guessing that it isn't intentional.

Removing that bug/feature also makes the code much simpler.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-14 10:49:44 -06:00
Ulf Magnusson
32e6565fd7 scripts/dts: Merge Bindings.binding() into load_bindings()
load_bindings() is trivial, and it allows some stuff to simplified, like
the check for no bindings being found, and the assignments to the
extract.globals.{bindings,bus_bindings,bindings_compat}.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-14 10:49:44 -06:00
Ulf Magnusson
706809e9b0 scripts/dts: Turn Bindings._included into a global
This is (probably) a temporary change just to be able to move some stuff
out of the Bindings class.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-14 10:49:44 -06:00
Ulf Magnusson
a1f1969331 scripts/dts: Separate out finding binding (.yaml) files
As a step in moving some functions out of the Bindings class.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-14 10:49:44 -06:00
Ulf Magnusson
d859f4f937 scripts/dts: Move initialization logic into load_bindings()
Makes main() more readable.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-14 10:49:44 -06:00
Ulf Magnusson
e17af82190 scripts/dts: Rename load_yaml_descriptions() to load_bindings()
More explicit, and consistent with other code.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-14 10:49:44 -06:00
Kumar Gala
1f1282afa4 scripts/dts/extract_dts_includes: Limit conf file to DT prefix defines
Only write out to the .conf file if the define starts with "DT_".  The
the conf file should only be used by sanitycheck, west, and
kconfigfunctions at this time.  In the future we should remove it, so
lets limit what's it exposing at this time.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-14 10:45:25 -06:00
Kumar Gala
b814af827d scripts/dts/extract: Fix how we determine DT_FLASH_AREA_<X>_DEV
DT_FLASH_AREA_<X>_DEV should be the label of the flash controller.  For
flashes that are "jedec,spi-nor" the flash itself is considered the
controller.  For "soc-nv-flash" the parent of the flash is assumed to be
the controller.

Rework the logic in how we determined DT_FLASH_AREA_<X>_DEV to handle
both cases.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-14 05:13:35 -06:00
Ulf Magnusson
4e02590231 scripts/dts: Change 'Error::' to 'Error:'
Typo.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-13 08:53:07 -06:00
Ulf Magnusson
b29a5b0e83 scripts/dts: Add a check_binding_properties() function
Removes some logic from merge_included_bindings() and makes it easier to
follow.

Also remove an outdated comment ('id' no longer exists) and some
comments that might not be helpful.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-13 08:53:07 -06:00
Ulf Magnusson
2b8766db53 scripts/dts: Improve documentation for merge_included_bindings()
Use a plain-English description.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-13 08:53:07 -06:00
Ulf Magnusson
3f8616add9 scripts/dts: Get rid of yaml_list in Bindings.bindings()
The yaml_list dictionary is three regular variables in disguise, as keys
are never added. Use regular variables instead, which is less confusing.

Hopefully the naming makes sense.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-13 08:53:07 -06:00
Ulf Magnusson
9e8c3fdf16 scripts/dts: Fix recursion in merge_included_bindings()
The consistency checks at the beginning of the function were not done
for "leaf" .yaml files.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-13 08:53:07 -06:00
Ulf Magnusson
6a15130b9a scripts/dts: Rename yaml_traverse_inherited()
...to merge_included_bindings().

This gives a better hint to what it does (merges properties from
!include'd files).

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-13 08:53:07 -06:00
Ulf Magnusson
00b2ec05e5 scripts/dts: Document Bindings._include()
Explain that it implements !include.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-13 08:53:07 -06:00
Ulf Magnusson
b98cdc10f6 scripts/dts: Have Bindings._include() always return a list
Makes yaml_traverse_inherited() more straightforward.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-13 08:53:07 -06:00
Ulf Magnusson
105c5b426b scripts/dts: Remove support for !import
Implemented as an undocumented and unused synonym for !include.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-13 08:53:07 -06:00
Ulf Magnusson
339db385ef scripts/dts: Simplify Bindings._include()
Use a list comprehension for the '!include [foo.yaml, bar.yaml]' case.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-13 08:53:07 -06:00
Ulf Magnusson
377a794378 scripts/dts: Remove dead '!include <dict>' implementation
Remove the '!include {foo: bar}' support from Bindings._include(). Only
'!include foo.yaml' and '!include [foo.yaml, bar.yaml]' are documented,
and yaml_traverse_inherited() doesn't implement the dict case either.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-13 08:53:07 -06:00
Sebastian Bøe
1ca1075b20 cmake: kconfig: pass GENERATED_DTS_BOARD_CONF on to kconfig
Make kconfigfunctions.py agnostic to where GENERATED_DTS_BOARD_CONF is
located.

We don't want to encode the path of GENERATED_DTS_BOARD_CONF into more
places than necessary. Also, re-using GENERATED_DTS_BOARD_CONF makes
it easier to change where generated_dts_board.conf is located.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2019-02-12 20:16:47 -05:00
Sebastian Bøe
cd4354317e kconfig: Have Kconfig fail when PROJECT_BINARY_DIR is not set
kconfigfunctions.py will silently set all values to '0' when the
environment variable 'PROJECT_BINARY_DIR' is not set.

But PROJECT_BINARY_DIR not being set indicates an internal error, so
we should instead error out with an appropriate error message.

Silently failing is dangerous and increases debug time. It is better
to fail fast.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2019-02-12 20:16:47 -05:00
Erwan Gouriou
7fce83ff8e kconfigfunctions: Disambiguate reference to DTS generated database
Documentation for kconfigfunctions, mentions they look up elements
in "the DTS generated "conf" style database". This indication
could be cryptic for new zephyr users. Adding the exact name and
path of the file for disambiguation.

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2019-02-12 11:37:53 -06:00
Alexander Wachter
8757e780fc drivers: can: Update userspace handlers
This commit updates the userspace handlers to thle latest version.

Signed-off-by: Alexander Wachter <alexander.wachter@student.tugraz.at>
2019-02-12 09:39:30 -05:00
Anas Nashif
062d05645b ci: tests: do not build all tests when docs/images change
When we change documentation or board images, do not build all tests for
this board. Only add more tests when actual code has changed.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-02-09 17:22:08 -06:00
Anas Nashif
115b811016 ci: do not build all tests when docs/images change
When we change documentation or board images, do not build all tests for
this board. Only add more tests when actual code has changed.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-02-09 17:22:08 -06:00
Kumar Gala
561d6dd1ed scripts/dts/extract_dts_includes: Mark non DT_ defines as deprecated
Mark any define that doesn't start with DT_, LED, SW, or have PWM_LED in
the name as deprecated via __DEPRECATED_MACRO.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-09 04:47:59 -06:00
Ulf Magnusson
5c67ebe4a8 scripts/dts: Clean up Bindings.bindings()
Some more work will be done here later (yaml_list could be replaced by a
few plain variables, for example), but this is a start:

 - Extract the compat string from the 'compatible:' line directly in the
   regex, instead of separately

 - Flatten the code, turning

     if a:
       ...
       if b:
         ...
         if c:
           ...

   into

     if not a:
        continue

     ...

     if not b:
        continue

     ...

     if not c:
        continue

     ...

 - Add some comments

 - Use clearer naming

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-08 22:57:05 -06:00
Ulf Magnusson
4e5868f4af scripts/dts: Make all_compats() return a set with all compats
Previously, all_compats() returned a dictionary mapping nodes to their
'compatible' values. This dictionary was then (only) used to generate a
set() of all 'compatible' strings.

Save a step and get rid of some code by having all_compats() return a
set() with all 'compatible' strings directly.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-08 22:57:05 -06:00
Ulf Magnusson
4fcdd134c2 scripts/dts: Improve naming in create_phandles()
k, v -> child_name, child_node

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-08 22:57:05 -06:00
Ulf Magnusson
8c515b25a3 scripts/dts: Remove unused create_phandles() 'handle' parameter
Unused.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-08 22:57:05 -06:00
Ulf Magnusson
ace8450040 scripts/dts: Simplify get_all_compatibles()
- Rename to all_compats(). The "compats" part is consistent with
   elsewhere.

 - Don't require an empty dictionary to be passed in. Build the
   result within the function instead.

 - Use more specific names (k, v -> child_name, child_node)

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-08 22:57:05 -06:00
Ulf Magnusson
cb507889dd scripts/dts: Replace dts['/'] with 'root'
Factors out some code.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-08 22:57:05 -06:00
Ulf Magnusson
b52290fa58 scripts/dts: Rename get_chosen() to create_chosen()
It creates the global 'chosen' dictionary. get_*() makes it sound like
it returns a value.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-08 22:57:05 -06:00
Ulf Magnusson
28c76fd3d8 scripts/dts: Rename get_aliases() to create_aliases()
It creates the global 'aliases' dictionary. get_*() makes it sound like
it returns a value.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-08 22:57:05 -06:00
Ulf Magnusson
6d824feb69 scripts/dts: Rename get_phandles() to create_phandles()
It creates the global 'phandles' dictionary. get_*() makes it sound like
it returns a value.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-08 22:57:05 -06:00
Ulf Magnusson
ccf7de0306 scripts/dts: Rename get_reduced() to create_reduced()
It creates the global 'reduced' dictionary. get_*() makes it sound like
it returns a value.

Fix a comment that might make people think the global dictionaries are
Python lists too.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-08 22:57:05 -06:00
Ulf Magnusson
9eb0d33ca0 scripts/dts: Simplify extract_string_prop()
Check if the node is missing from 'defs' first, and insert it if not.
That way, later code can assume it exists.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-08 22:57:05 -06:00
Kumar Gala
3de6de305f west: sign: Convert FLASH_WRITE_BLOCK_SIZE to DT_
Use DT_FLASH_WRITE_BLOCK_SIZE prefixed defined instead of
FLASH_WRITE_BLOCK_SIZE as the non-DT version is deprecated.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-08 22:39:07 -06:00
Kumar Gala
65cd010fc1 west: sign: Convert FLASH_AREA to DT_FLASH_AREA
Use DT_FLASH_AREA prefixed defined instead of FLASH_AREA as the non-DT
version is deprecated.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-08 22:39:07 -06:00
Kumar Gala
ec63dca3ea scripts/dts/extract: Fix producing bogus alias if reg has no size
Its possible that a reg property is address only.  If that's the case we
should produce any alias information for the size.  We should check
nr_size_cells to decide to handle the alias case or not.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-08 22:39:07 -06:00
Kumar Gala
e3e87f624f scripts/dts/extract_dts_includes: Hand alias to alias for write_conf
If we have an alias pointing to another alias we would get None in the
resulting conf file.  However that alias might exist, so try looking it
up again in the aliases db.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-08 22:39:07 -06:00
Kumar Gala
5a20a418a2 scripts/dts/extract: Remove generation for FLASH_LABEL
FLASH_LABEL define isn't used anywhere so remove it.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-08 22:39:07 -06:00
Kumar Gala
873e4e950a scripts/dts/extract_dts_includes: Add DT_ prefix to FLASH_AREA defines
We generate the following FLASH prefixed defines:

	FLASH_AREA_MCUBOOT_LABEL
	FLASH_AREA_MCUBOOT_OFFSET
	FLASH_AREA_MCUBOOT_OFFSET_0
	FLASH_AREA_MCUBOOT_READ_ONLY
	FLASH_AREA_MCUBOOT_SIZE
	FLASH_AREA_MCUBOOT_SIZE_0

Now we generate DT_ prefixed versions:

	DT_FLASH_AREA_MCUBOOT_LABEL
	DT_FLASH_AREA_MCUBOOT_OFFSET
	DT_FLASH_AREA_MCUBOOT_OFFSET_0
	DT_FLASH_AREA_MCUBOOT_READ_ONLY
	DT_FLASH_AREA_MCUBOOT_SIZE
	DT_FLASH_AREA_MCUBOOT_SIZE_0

And will deprecate the non DT_ prefixed versions in the future.  We keep
them around as aliases for the time being.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-08 22:39:07 -06:00
Kumar Gala
2024f6b2a0 scripts/dts/extract_dts_includes: Add DT_ prefix to FLASH defines
We generate the following FLASH prefixed defines:
	FLASH_LABEL
	FLASH_ERASE_BLOCK_SIZE
	FLASH_WRITE_BLOCK_SIZE

Now we generate DT_ prefixed versions:
	DT_FLASH_LABEL
	DT_FLASH_ERASE_BLOCK_SIZE
	DT_FLASH_WRITE_BLOCK_SIZE

And will deprecate the non DT_ prefixed versions in the future.  We keep
them around as aliases for the time being.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-08 22:39:07 -06:00
Torsten Rasmussen
7e9d1bdda4 CMake/Kconfig: CMake and Kconfig build integration
This commit allows for Zephyr modules to be natively integrated into
the build system with CMakeLists.txt and Kconfig files.

The sourcing of module files are done in following order:
- If <module>/zephyr/module.yml exists, use cmake and kconfig settings
  for sourcing of additional file
- Else if <module>/zephyr/CMakeLists.txt exists, source this file into
  CMake build tree and add <module>/zephyr/Kconfig as osource

If none of the above files are present, the project is considered to
not be a Zephyr module

Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
2019-02-08 22:47:02 -05:00
David B. Kinder
1cc8bbb4ae doc: fix misspelling in kconfig doc
Fix typos introduced by PR #13186

Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
2019-02-08 16:33:09 -06:00
Kumar Gala
c7606609be dts: Remove base_label support
We've removed all references to base_label in dts bindings and are
removing the functionality.  We should depend on DT_ prefixed defines
going forward.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-08 14:44:42 -06:00
Carles Cufi
acc82cee00 scripts: west: Tweak west build doc
Clarify what "being in a build directory" exactly means.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-02-08 14:58:28 -05:00
Kumar Gala
f2ef52f122 kconfig: kconfigfunctions: update dt_str_val
Clarify the docs for dt_str_val that if the name isn't found we return
and empty string.  Also cleanup the code slightly as we don't need to
escape the double quote.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-08 12:02:18 -06:00
Kumar Gala
ff70b3444f dts: Convert CONFIG_ to DT_ symbols for chosen props
Replace generating CONFIG_ symbols with DT_ symbols for chosen
properties like 'zephyr,console' or 'zephyr,bt-mon-uart'.  We now use a
kconfigfunctions (dt_str_val) to extract the info from dts into Kconfig.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-08 11:41:26 -06:00
Kumar Gala
2579adea1e kconfig: kconfigfunctions: Add dt_str_val function
Add dt_str_val to extract a string from the dt conf database.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-08 10:29:57 -06:00
Kumar Gala
bfaaa6bbe9 dts: Convert CONFIG_CCM to DT_CCM
Since we know do DTS before Kconfig we should try and remove dts from
creating Kconfig namespaced symbols and leave that to Kconfig.  So
rename CONFIG_CCM_<FOO> to DT_CCM_<FOO>.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-08 10:29:57 -06:00
Marti Bolivar
5591ca2374 sanitycheck: various fixes for device testing
Device testing has various problems:

- the main thread doesn't halt the serial monitoring thread when an
  error occurs while flashing, meaning flash errors can look like
  timeouts

- serial_line is unbound if the ser.readline() call raises an
  exception, so the following "if serial_line" would raise NameError
  in that case

- the serial monitoring thread is not a daemon, so exceptions or
  errors in the handle() thread that cause thread exit will leave the
  process sitting around until the serial monitoring thread has a
  chance to exit

Fix these issues:

- Rewrite the serial monitoring thread as a select loop over the
  serial port file descriptor and a pipe shared by the main thread and
  serial monitoring thread. If flashing fails, the main thread uses
  the pipe to signal to the serial monitoring thread that it should
  exit.

- Make sure serial_line is always bound when read.

- Mark the serial thread daemonic.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-02-08 08:08:32 -05:00