Commit Graph

106 Commits

Author SHA1 Message Date
Ulf Magnusson
d67095ba59 Kconfiglib: Make header symbol order match .config files again
Commit f6bf897780 ("kconfiglib: Add preprocessor and two warnings")
accidentally made the ordering of #define's in generated header files
random (dependent on Python's set() implementation). Minimal
configuration files (which can be saved from the menuconfig) were
affected too.

The intent is for the order to match .config files, which in turn match
symbol definition order. This doesn't affect behavior in any way, but
makes things more readable (and is handier if files are checked in).

Update Kconfiglib to upstream revision a3252f620fad9 to make the
ordering match again. Tests have been added upstream.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-08-22 15:29:18 -07:00
Ulf Magnusson
a56be6f58d Kconfiglib: Fix $srctree behavior for the top-level Kconfig file
Due a design goof, the top-level Kconfig file passed to
Kconfig.__init__() wasn't looked up relative to $srctree. This broke a
planned fix for an issue with the CI check for references to undefined
Kconfig files in external projects.

Update Kconfiglib to upstream revision 953cc12464 to fix it.

Also update doc/CMakeLists.txt to pass 'Kconfig' instead of '../Kconfig'
to genrest.py. $srctree is already set to $ZEPHYR_BASE.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-08-18 10:38:52 +02:00
Ulf Magnusson
6686efb838 kconfiglib/menuconfig: Add functionality and improve symbol information
Update Kconfiglib and menuconfig to upstream revision 6aea8d06b637e.

Kconfiglib changes:

 - Kconfig files are now looked up just relative to $srctree (if set).
   Previously, each source'd Kconfig file was also looked up in the
   current directory, and Kconfig files there could override Kconfig
   files in $srctree. This is what the C tools do.

   I'm pretty sure that behavior was a bug in the C tools all along, and
   only meant for .config files. It caused problems (and an ugly
   workaround) for the undefined Kconfig symbol CI check in an external
   project.

   The new behavior also saves a bunch of open()'s, though it's probably
   not noticeable.

 - Setting the KCONFIG_STRICT environment variable to 'y' now makes
   Kconfiglib itself warn for references to undefined symbols. This
   isn't safe in general, as some projects use multiple Kconfig trees
   with shared Kconfig files (e.g. the Linux kernel).

   This will be used to simplify the undefined Kconfig symbol CI check.

 - It's now possible to customize how symbols and choices are printed
   within expressions.

   This will be used to make the RST link generation in genrest.py less
   hacky.

 - Instead of having 'gsource', a plain 'source' is now globbing, and
   requires at least one match. There's also 'osource', for when it's
   okay for a glob pattern to match no files.

   'gsource' had the design flaw that there was no way to require at
   least one file to match. I plan on replacing all 'gsource' statements
   with plain 'source's later, but 'gsource' is still supported for
   backwards compatibility.

 - def_int, def_hex, and def_string are now available as a Kconfig
   extensions, analogous to def_bool (set type and add default).

 - Misc. internal cleanup.

menuconfig changes:

 - Boolean value hints are no longer shown to the right of defaults in
   the symbol information for int/hex symbols. Stuff like
   '- 74 (value: "n")' wasn't helpful, and looked confusing.

 - Symbol information has been made more compact in general, e.g. by
   skipping value hints where they aren't helpful
   ('FOO(=y)' instead of 'FOO(=y) (=y)'), and by shortening stuff like
   '(value: "y")' to just '(=y)'.

 - Misc. internal cleanup.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-08-13 19:31:46 -07:00
Ulf Magnusson
ec3eff57e0 Kconfig: Use the first default with a satisfied condition
Up until now, Zephyr has patched Kconfig to use the last 'default' with
a satisfied condition, instead of the first one. I'm not sure why the
patch was added (it predates Kconfiglib), but I suspect it's related to
Kconfig.defconfig files.

There are at least three problems with the patch:

  1. It's inconsistent with how Kconfig works in other projects, which
     might confuse newcomers.

  2. Due to oversights, earlier 'range' properties are still preferred,
     as well as earlier 'default' properties on choices.

     In addition to being inconsistent, this makes it impossible to
     override 'range' properties and choice 'default' properties if the
     base definition of the symbol/choice already has 'range'/'default'
     properties.

     I've seen errors caused by the inconsistency, and I suspect there
     are more.

  3. A fork of Kconfiglib that adds the patch needs to be maintained.

Get rid of the patch and go back to standard Kconfig behavior, as
follows:

  1. Include the Kconfig.defconfig files first instead of last in
     Kconfig.zephyr.

  2. Include boards/Kconfig and arch/<arch>/Kconfig first instead of
     last in arch/Kconfig.

  3. Include arch/<arch>/soc/*/Kconfig first instead of last in
     arch/<arch>/Kconfig.

  4. Swap a few other 'source's to preserve behavior for some scattered
     symbols with multiple definitions.

     Swap 'source's in some no-op cases too, where it might match the
     intent.

  5. Reverse the defaults on symbol definitions that have more than one
     default.

     Skip defaults that are mutually exclusive, e.g. where each default
     has an 'if <some board>' condition. They are already safe.

  6. Remove the prefer-later-defaults patch from Kconfiglib.

Testing was done with a Python script that lists all Kconfig
symbols/choices with multiple defaults, along with a whitelist of fixed
symbols. The script also verifies that there are no "unreachable"
defaults hidden by defaults without conditions

As an additional test, zephyr/.config was generated before and after the
change for several samples and checked to be identical (after sorting).

This commit includes some default-related cleanups as well:

  - Simplify some symbol definitions, e.g. where a default has 'if FOO'
    when the symbol already has 'depends on FOO'.

  - Remove some redundant 'default ""' for string symbols. This is the
    implicit default.

Piggyback fixes for swapped ranges on BT_L2CAP_RX_MTU and
BT_L2CAP_TX_MTU (caused by confusing inconsistency).

Piggyback some fixes for style nits too, e.g. unindented help texts.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-08-10 12:38:28 -07:00
Ulf Magnusson
5402662dd9 cmake: kconfig: Fix rerunning cmake after Kconfig warnings
Commit b3d165f ("scripts: kconfig: Handle warnings generated
during evaluation") made it common for kconfig.py to fail after writing
zephyr/.config. This confuses the configuration fragment checksum logic
in cmake/kconfig.cmake, because it expects the saved checksum file to
exist if zephyr/.config exists.

The end result is a CMake error when rerunning the configuration after
non-whitelisted Kconfig warnings.

Fix it by only writing zephyr/.config (and zephyr/include/autoconf.h) in
kconfig.py if there are no warnings-turned-errors.

Also check if the saved checksum file exists in kconfig.cmake before
trying to open it. Normally this shouldn't happen though.

Move the writing of the checksum file to before writing zephyr/.config
as well. That way, zephyr/.config only gets written if the other
operations succeed.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-08-02 19:15:26 +02:00
Ulf Magnusson
b3d165f3c2 scripts: kconfig: Handle warnings generated during evaluation
Warnings generated during symbol evaluation were accidentally ignored,
due to checking for warnings before writing .config and autoconf.h
(which indirectly evaluates all symbols).

Move the warning checking code to after writing the configuration to
catch such warnings. kconfig.py still gets rerun if any
warnings-turned-errors show up.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-07-31 08:26:28 -04:00
Anas Nashif
9930633454 kconfig: remove whitelisting for ARC_INIT
ARC_INIT was whitelisted in kconfig, removing now that the issue is
fixed.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-07-30 15:17:51 -04:00
Ulf Magnusson
953cc12464 kconfiglib: Fix paths for gsource'd files in the documentation
$srctree was changed to an absolute path when the documentation building
was switched over to CMake, which uncovered a bug in Kconfiglib that
caused symbols in gsource'd files to show up with absolute paths in the
auto-generated Kconfig documentation.

This commit adds upstream commit ac692af07a123 ("Fix absolute $srctree
prefixes showing up on gsource'd files"), which fixes it.

Upstream commit message:

  When using gsource with $srctree set to an absolute path, the $srctree
  prefix would show up in MenuNode.filename, trickling its way into e.g.
  generated documentation.

  This was due to a broken test: os.path.isabs() was checked after
  joining the pattern with $srctree, making it mistake an absolute
  $srctree for an absolute path in the Kconfig file.

  Fix the test.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-07-14 08:17:58 +02:00
Ulf Magnusson
f6bf897780 kconfiglib: Add preprocessor and two warnings
Update Kconfiglib to upstream revision 547fced630611 to get a new
Kconfig preprocessor in, documented in
https://github.com/torvalds/linux/blob/master/Documentation/kbuild/kconfig-macro-language.txt.

The preprocessor allows shell functions to be run directly from Kconfig.
Things like 'default "prefix-$(shell,some-cmd)' and
'depends on (success,some-cmd)' are supported.

The preprocessor might come in handy for Kconfig/DTS integration. I'm
thinking of extending it so that Python functions can be called as well.

There's also two new warnings:

 - Trying to use an int/hex symbol like a bool symbol in an expression
   (where it will always evaluate to n)

 - Having a 'default' on an int/hex symbol that lies outside an active
   'range'.

The parser is more strict now as well (due to dropping some hacks for 3+
year old kernel versions).

A related fix for scripts/kconfig.py is also included:

The comment above the whitelist lies. I accidentally changed the warning
text for the select-with-unsatisfied-dependencies warning while
generalizing it (for m-valued dependencies, which you'd never get in
Zephyr).

Update the whitelist to detect the new warning text.

Last-minute piggyback:

Include a change that improved the parse time for U-Boot from 4 seconds
to about 0.6 seconds, related to symbols defined in multiple locations.
It might be helpful for Zephyr as well, as it also uses a lot of symbols
defined in multiple locations.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-07-11 17:02:22 +02:00
Ulf Magnusson
53f4189075 scripts: kconfig: Do not print warnings for choice overriding
Andy Ross wanted to override a choice selection from a board defconfig
file in a prj.conf, but this could trigger the
assigned-value-didn't-take warning, because the choice symbol set to y
in the board defconfig file ends up as n after another choice symbol is
selected.

(Note: Setting any choice symbol to y is enough to make it the choice
(user) selection. There's no need to set the other symbols to n.)

Fix the warning by checking choices at the choice level rather than at
the level of individual choice symbols. This also makes the warning a
bit more informative in general for choice symbols.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-07-03 17:09:47 -04:00
Ulf Magnusson
aa2beb9f10 kconfig: Stop whitelisting "undefined symbol SSE" warning
The warnings were fixed by commit c4123643b5 ("tests: fp_sharing:
Extract x86 configs to separate .conf").

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-06-27 17:07:16 +02:00
Ulf Magnusson
d317a0e6b4 kconfiglib: Update to use redesigned 'referenced' API
Update Kconfiglib to get upstream commit eb6c21a9b33a2 ("Turn
MenuNode/Symbol/Choice.referenced() into a @property") in. It converts
the MenuNode.referenced() function into a property, which makes the API
more consistent (read-only stuff uses properties).

Also update scripts/ci/list_undef_kconfig_refs.py to access .referenced
as a property.

Piggyback a small is_num() simplification.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-06-25 10:31:08 +02:00
Ulf Magnusson
54a5997f5c kconfiglib: Add dependency loop detection
Update Kconfiglib to get upstream commit ca89ca0c0c420 ("Add dependency
loop detection") in.

Upstream commit message
=======================

Pretty long overdue.

Until now, dependency loops have raised a hard-to-debug Python
RecursionError during evaluation. A Kconfiglib exception is raised now
instead, with a message that lists all the items in the loop.

See the comment at the start of _check_dep_loop_sym() for an overview of
the algorithm. At a high level, it's loop detection in a directed graph
by keeping track of unvisited/visited nodes during depth-first search.
(A third "visited, known to not be in a dependency loop" state is used
as well.)

Choices complicate things, as they're inherently loopy: The choice
depends on the choice symbols and vice versa, and the choice symbols in
a sense all depend on each other.

Add the choice-to-choice-symbol dependencies separately after dependency
loop detection, so that there's just the choice-symbol-to-choice
dependencies to deal with. It simplifies things, as it makes it possible
to tell dependencies from 'prompt' and 'default' conditions on the
choice from choice symbol dependencies.

Do some flag shenanigans to prevent the choice from being "re-entered"
while looping through the choice symbols. Maybe this could be cleaned up
a bit somehow...

Example exception message:

Dependency loop
===============

A (defined at tests/Kdeploop10:1), with definition...

config A
        bool
        depends on B

...depends on B (defined at tests/Kdeploop10:5), with definition...

config B
        bool
        depends on C = 7

...depends on C (defined at tests/Kdeploop10:9), with definition...

config C
        int
        range D 8

...depends on D (defined at tests/Kdeploop10:13), with definition...

config D
        int
        default 3 if E
        default 8

...depends on E (defined at tests/Kdeploop10:18), with definition...

config E
        bool

(select-related dependencies: F && G)

...depends on G (defined at tests/Kdeploop10:25), with definition...

config G
        bool
        depends on H

...depends on the choice symbol H (defined at tests/Kdeploop10:32), with
definition...

config H
        bool
        prompt "H" if I && <choice>
        depends on I && <choice>

...depends on the choice symbol I (defined at tests/Kdeploop10:41), with
definition...

config I
        bool
        prompt "I" if <choice>
        depends on <choice>

...depends on <choice> (defined at tests/Kdeploop10:38), with
definition...

choice
        bool
        prompt "choice" if J

...depends on J (defined at tests/Kdeploop10:46), with definition...

config J
        bool
        depends on A

...depends again on A (defined at tests/Kdeploop10:1)

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-06-20 15:53:38 -04:00
Ulf Magnusson
4dcde2e628 menuconfig: Allow searches from the info dialog and vice versa
Having to go back to the main display all the time gets awkward,
especially when searching to look up symbol information.

Allow the symbol information dialog to be opened from the search dialog
(with Ctrl-F, since '?' is already used there as a regex metacharacter),
and also allow a search to be started from the symbol information
dialog.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-06-18 14:41:53 -04:00
Ulf Magnusson
80f19cca42 kconfiglib: Correctly report choice locations in some warnings
This commit gets upstream commit dc0b022247b85 ("Correctly report choice
locations in some warnings") in, which fixes some warnings that would
previously report the location of a choice as being "undefined" (which
is impossible).

Upstream commit message:

  Menu nodes were added to choices after parsing their properties,
  making some warnings generated during parsing (as opposed to in
  _check_choice_sanity()) incorrectly give the choice as '<choice>
  (undefined)'.

  Add the node before parsing choice properties to fix those warnings.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-06-18 14:41:53 -04:00
Ulf Magnusson
ea108107e6 scripts: kconfig: Extend the assignment-failed warning
- Tailor the warning when the symbol has no prompt, explaining how
   promptless symbols get values. Add some anti-select propaganda too.

 - Reference the 'Setting configuration values' in the Board Porting
   Guide. It explains Kconfig.defconfig files.

Fixes: #8388

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-06-18 12:49:12 -04:00
Ulf Magnusson
59c8ae8caf kconfiglib: Fix incorrectly ordered props. for some multi.def symbols
This commit fixes a pretty nasty bug that could cause properties on
symbols and choices defined in multiple locations to end up in the wrong
order, potentially affecting evaluation.

Alexander Wachter ran into this for an out-of-tree build.

Multi.def. symbols are rare in the Linux kernel, which is what the
Kconfiglib test suite uses for compatibility testing, so this managed to
slip through. Comprehensive selftests have been added for property
ordering on nested multi.def. symbols/choices.

This bug was introduced by commit e307ba340c ("kconfiglib: Record
which MenuNode has each property").

Commit message from Kconfiglib (c8801514d63aa)
==============================================

Fix incorrectly ordered properties for some nested multi.def. symbols

_propagate_deps() visits menu nodes roughly breadth-first, meaning
properties on symbols and choices defined in multiple locations could
end up in the wrong order when copied from the menu node for some
unlucky if/menu nestings.

Fix it by moving the menu-node-to-symbol/choice property copying in
_finalize_tree() so that it's guaranteed to happen in definition order.

This bug was introduced by commit 63a4418 ("Record which MenuNode has
each property").

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-06-14 19:45:58 +02:00
Ulf Magnusson
f425c0aa27 scripts: kconfig: Disable the "FOO set more than once" warning
Some prj.conf files seem to deliberately override settings from the
board configuration (e.g. samples/bluetooth/hci_usb/prj.conf, with
GPIO=y). Disable the warning about a symbol being assigned more than
once to avoid warnings for those cases.

A list similar to WARNING_WHITELIST could be added later if more
warnings need to be disabled.

Also refactor the warning checking code a bit to get rid of some not's.

Suggested by Sebastian Bøe.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-06-13 14:06:38 +02:00
Tomasz Gorochowik
7f84001fd3 menuconfig: Fix searching for nonexistent objects
This commit fixes an issue when user searches for a nonexistent object
(e.g. adsdsaasda) and presses enter.

Having all the key checks in one continuous if statement makes sure that
the very last 'else' statement does not get executed when enter is
pressed.

Signed-off-by: Tomasz Gorochowik <tgorochowik@antmicro.com>
2018-06-12 20:27:20 -04:00
Ulf Magnusson
f971aacaf3 scripts: kconfig: Turn most warnings into errors
In particular, this will turn assignments to undefined Kconfig variables
into errors, which are very easy to miss otherwise (e.g. when Kconfig
symbols get renamed or removed).

Warnings generated by anything tested by CI (scripts/sanitycheck) will
be caught.

Have a whitelist of warnings that are not turned into errors. Some
warnings currently whitelisted should be turned into errors as well, but
would require a bit of work.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-06-12 20:18:14 -04:00
Ulf Magnusson
20721f39fa scripts: kconfig: Improve the 'user value != actual value' warning
Symbols that are assigned values in .config files must have satisfied
dependencies, and must have a prompt. Otherwise, the assigned value is
ignored. A warning is printed if the symbol ends up with a different
value than the assigned value as a result.

It might be difficult to know how to fix the problem just from seeing
the current warning. Add some hints to it to help out:

  - The symbol information dialog in menuconfig is good for figuring out
    dependencies that need to be enabled. Mention menuconfig in the
    warning.

  - The page for the symbol in the autogenerated Kconfig docs can be
    helpful too, so link it. There's a slight chance that it'll be
    outdated, but it's usually correct when working on the master
    branch.

Automatically enabling dependencies is much trickier than it might seem
at first, due to the generality of Kconfig. See
https://github.com/zephyrproject-rtos/zephyr/issues/8181 for some
discussion.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-06-11 17:24:07 -04:00
Ulf Magnusson
cb95ea0b53 kconfiglib: Update to add list of choices
Update Kconfiglib to upstream revision 9fba375c65341 (+ local Zephyr
modifications) to get commit 94020beb311eb ("Make Kconfig._choices
public") in. It will be used to generate Kconfig reference pages for
choices.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-06-11 14:40:48 -04:00
Ulf Magnusson
163dec6dc3 menuconfig: Add show-name mode + small help dialog improvement
- Pressing 'c' ('n' was taken) toggles a mode where the names of all
   symbols are shown before their prompts. This saves going into the
   help display when you just want the name.

 - The name of the symbol/choice now appears in the main text of the
   help dialog rather than in the title, where it might be hard to
   spot, as pointed out by Carles Cufi.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-05-28 21:04:34 +02:00
Ulf Magnusson
6eabea3a7e Kconfiglib: Warn for unquoted string defaults
Unquoted string defaults work through a quirk of Kconfig (undefined
symbols get their name as their string value), but look confusing. It's
done inconsistently now too.

Suggested by Kumar Gala.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-05-26 19:17:48 -04:00
Ulf Magnusson
d44fee359d menuconfig: Add search and save/load improvements from upstream
- Show the value of each symbol in the jump-to dialog.

 - Search names and prompts separately in the jump-to dialog.
   Previously, '_BAR$' wouldn't match FOO_BAR if it had a prompt,
   because '_BAR$' was matched against the string 'FOO_BAR "prompt"'.

 - Show the working directory in save/load dialog boxes. Paths will be
   relative to it, and Carles Cufi pointed out that it might not be
   obvious.

   Also allow ~ to be used to refer to the home directory.

 - Implement scroll offset for edit boxes. This makes it clearer when
   they're scrolled horizontally.

Piggyback an update of Kconfiglib to the latest version (no functional
changes).

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-05-24 17:41:45 +02:00
Ulf Magnusson
b45e152ff0 menuconfig/genrest: Exclude implicit submenus from menu paths
Make all menu paths ("(top menu) -> menu -> submenu -> ...") exclude
implicit submenus, which are shown indented in the menuconfig interface
and are created when items depend on the symbol before them.

Previously, implicit submenus were excluded in the menu path at the top
of the menuconfig interface, but were included in the menuconfig symbol
information dialog and in the docs generated by genrest.py.

This makes it consistent, and un-spams the documentation for some
symbols a bit.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-05-22 13:31:56 -04:00
Ulf Magnusson
aa26289458 kconfig: Get rid of leading/trailing whitespace in prompts
Leading/trailing whitespace in prompts requires ugly workarounds in
genrest.py, as e.g. *prompt * is invalid RST. strip() all prompts in
Kconfiglib and get rid of the genrest.py workarounds. Add a warning too.

The Kconfiglib update has some unrelated cleanups and fixes (that won't
affect Zephyr).

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-05-19 09:26:39 +03:00
Ulf Magnusson
b58cbfd6ab menuconfig: Add .config loading dialog
.config files can now be loaded from within the menuconfig interface.
Show-all mode is turned on if the selected menu entry becomes invisible.

Unrelated fix: The menu path at the top of the display no longer
includes implicit (indentation-based) submenus, which makes it a bit
less spammy and confusing.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-05-18 20:17:52 +03:00
Ulf Magnusson
6d4d8ce026 menuconfig: Show properties on the correct symbol definition
This commit mirrors 0f1229bd68866 ("doc: genrest: Show properties on the
correct symbol definition"), for the menuconfig symbol information
display.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-05-17 23:55:07 +03:00
Ulf Magnusson
e307ba340c kconfiglib: Record which MenuNode has each property
This commit does some major surgery to Kconfiglib so that properties
(defaults, selects, etc.) can be shown on the menu node that actually
has the propetrty for symbols and choices defined in multiple locations.

This will be used to improve the output of genrest.py and the symbol
information display in the menuconfig.

The parsing code is a bit simpler now too as a side effect.

Commit message from Kconfiglib (63a44186137e2)
==============================================

This allows accurate documentation to be generated for symbols and
choices defined in multiple locations. There are now MenuNode.defaults,
MenuNode.selects, etc., lists that mirror the corresponding
Symbol/Choice lists.

Symbol/Choice.__str__() now correctly show property locations as well,
by simply concatenating the strings returned by MenuNode.__str__() for
each node.

_parse_properties() was modified to add all properties directly to the
menu node instead of adding them to the contained symbol or choice. The
properties are then copied up to symbols and choices in
_finalize_tree(). Dependency propagation is handled at the same time.

As a side effect, this cleans up the code a bit and de-bloats
_parse_properties().

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-05-17 23:55:07 +03:00
Ulf Magnusson
4dc9e5b2de kconfig: Get rid of 'option env' bounce symbols
This commit gets rid of the 'option env="ENV_VAR"' bounce symbols.
"$FOO" now expands directly to the value of the environment variable
FOO, instead of to the value of the Kconfig symbol FOO.

This change is likely to soon appear in the C tools as well. Those
'option env' symbols always seemed kinda pointless, and have broken
dependency handling due to forcing symbol evaluation during parsing,
before all the symbols have even been seen.

Compatibility with the C tools could be retained by naming all
'option env' symbols the same as the environment variable they
reference.

This commit also updated the Zephyr documentation to explain the new
behavior. It's relevant for $ZEPHYR_BASE and out-of-tree Kconfig
extensions.

Commit message from Kconfiglib (cbf32e29a130d)
==============================================

Make "$FOO" directly reference the environment variable $FOO in e.g.
'source' statements, instead of the symbol FOO. Use os.path.expandvars()
to expand strings (which preserves "$FOO" as-is if no environment
variable FOO exists).

This gets rid of the 'option env' "bounce" symbols, which are mostly
just spam and are buggy in the C tools (dependencies aren't always
respected, due to parsing and evaluation getting mixed up). The same
change will probably appear soon in the C tools as well.

Keep accepting 'option env' to preserve some backwards compatibility,
but ignore it when expanding strings. For compatibility with the C
tools, bounce symbols will need to be named the same as the environment
variables they reference (which is the case for the Linux kernel).

This is a compatibility break, so the major version will be bumped to 6
at the next release.

The main motivation for adding this now is to allow recording properties
on each MenuNode in a clean way. 'option env' symbols interact badly
with delayed dependency propagation.

Side note: I have a feeling that recording environment variable values
might be redundant to trigger rebuilds if sync_deps() is run at each
compile. It should detect all changes to symbol values due to
environment variables changing value.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-05-17 23:55:07 +03:00
Ulf Magnusson
e24788eb71 menuconfig: Make search more flexible and search prompts
This commit gets the following incremental search improvements in from
upstream:

  - 1d3db5de9b8c2 ("menuconfig: Add search with multiple search
    strings")

    This makes a search string like 'foo bar' match all symbol names
    that match both 'foo' and 'bar' (which can be regexes), regardless
    of the order in which they appear in the match. This is faster and
    more flexible than having to type a bunch of '.*'.

  - 9bf8fc6e6907e ("menuconfig: Add prompts to incremental search")

    This makes the incremental searcher search prompt strings as well as
    symbol names.

    The prompt is now displayed next to the symbol name in the list of
    matches as well.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-05-11 16:44:47 -04:00
Ulf Magnusson
295c1d8580 menuconfig: Fix rendering of long prefilled edit box string
The horizontal scroll (hscroll) wasn't initialized properly when the
initial (prefilled) contents of an edit box was longer than the edit box
itself (e.g. when saving with a long path in KCONFIG_CONFIG). Things
snapped back into place once a key was pressed.

Properly initialize hscroll to fix the initial rendering.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-05-10 23:27:17 +02:00
Ulf Magnusson
27d34926e5 menuconfig: Increase indent and make Unicode more robust
This commit adds the following changes from upstream:

  - ed38e895ace ("Increase indent for implicit submenus to 4")

    Suggested by Carles Cufí. Makes it easier to see the menu structure
    at a glance.

  - 1d252b30c77 ("menuconfig: Convert the C locale to a UTF-8 locale for
                  LC_CTYPE")

    Makes Unicode input work on many systems with bad defaults.

    Also fixes some interface ugliness, like down arrows turning into
    upside-down T's.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-05-09 14:08:00 +02:00
Ulf Magnusson
547ed9b563 kconfig: Make 'source' non-globbing and use 'gsource'
Until now, Zephyr has used a patched Kconfiglib that turns 'source' into
a globbing source (by replacing 'source' with 'gsource' at the token
level). There's two problems with this:

  - The patch needs to be maintained separately

  - Misspelled filenames are silently ignored, as they look like glob
    patterns that don't match anything

Fix it as follows:

  1. Replace all 'source' statements that use wildcards with 'gsource'

  2. Remove the custom Kconfiglib patch so that 'source' no longer globs

The sed pattern '/source.*[*?]/s/source/gsource/' was run over all
Kconfig* files to do the replacement.

source's that use environment variables that might contain glob patterns
were manually changed to gsource.

Building the docs in doc/ is a good test, as doc/Makefile deliberately
sets the environment variables to glob up as many Kconfig files as
possible.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-05-08 11:14:12 +02:00
Ulf Magnusson
ac7f223956 kconfig: Mention that checkconfig.py lacks Kconfiglib 2 support
This script has not been updated for Kconfiglib 2. Add a comment to it
that mentions it.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-05-07 21:53:37 +02:00
Ulf Magnusson
11952a60bf kconfig: Remove the C Kconfig implementation
Remove the C Kconfig tools and various scripts associated with them.

scripts/kconfig/diffconfig is popular, so keep it.

I don't know whether anyone is using scripts/kconfig/config. Remove it
and see if anyone screams.

scripts/kconfig/streamline_config.pl deals with modules ('m' values) and
can safely be removed. Zephyr's Kconfig files do not use modules.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-05-07 21:53:37 +02:00
Ulf Magnusson
b737fcb9ba scripts: kconfig: Add incremental search to menuconfig
Pressing [/] brings up a dialog with an edit box where a regex can be
entered. The list of matching symbols is always shown below it.
Selecting a symbol and pressing [Enter] jumps directly to it in the menu
tree. If the symbol is invisible, show-all mode is turned on
automatically.

This commit also includes a bunch of more-or-less unrelated changes from
poking around with the code:

  - Some redundant styles were merged. Probably wouldn't want to have a
    different style for each separator line, for example...

  - [ESC] in the top menu now works like [Q]

  - Returning to a parent menu now makes sure that the selected row is
    visible, even if the terminal was shrunk between entering the child
    menu and leaving it.

  - A _max_scroll() helper was factored out to reduce code duplication.
    It takes a list of items and a window in which the list is
    displayed, with one row per item, and returns the minimum scroll
    value that will make the final item visible.

  - The save dialog now pops up a message to confirm that the save was
    successful.

  - Lots of minor code nits all over (renamings, etc.)

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-05-07 21:53:37 +02:00
Sebastian Bøe
60b01f3f54 kconfig: Refactor kconfig.py to use __main__ and argparse
Kconfig.py is not following the de-facto (real?) coding standards of
Zephyr. This commit refactors kconfig.py with two changes:

Use __main__ and def main().

Use argparse instead of sys.argv.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2018-05-04 17:25:14 -04:00
Ulf Magnusson
dc97fc2a60 kconfiglib: Update to default to UTF-8 for Python 3
Update Kconfiglib to upstream revision da40c014398f3 (+ local Zephyr
modifications) to get commit da40c014398f3 ("Force encoding to UTF-8 by
default on Python 3") in. It sets a (configurable) UTF-8 default for
Python 3, overriding the encoding specified in the current locale.

I've decided that this is a good idea after some problem reports
unrelated to Zephyr. Running with the C locale breaks things horribly
otherwise, and the fix isn't obvious.

Plain strings aren't decoded on Python 2, so no changes are needed
there.

Related PEP: https://www.python.org/dev/peps/pep-0538/

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-05-02 15:44:45 -04:00
Ulf Magnusson
1799cfdb2f scripts: kconfig: Turn malformed .config lines into errors
The warning from Kconfiglib might be hard to spot, and the problem is
easily fixed right away (note that malformed .config lines are different
from assignments to undefined symbols).

Do not make malformed .config lines an error in Kconfiglib itself (just
a warning), as you end up with messy "half-loaded" configurations.

Suggested by Marti Bolivar.

Piggyback some minor cleanups in kconfig.py. Make the warning for
configuration settings that didn't match the final value go to stderr,
for consistency.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-05-01 20:53:40 +02:00
Ulf Magnusson
cfb3c9251c kconfiglib: Update to add warning for malformed .config lines
Update Kconfiglib to upstream revision ed3ceaa056262 (+ local Zephyr
modifications) to get commits c1c5ef2eb1009 ("Print a warning for
malformed .config lines") and ed3ceaa05626f ("Make warnings available in
a list") in.

This warning for malformed .config lines will be turned into an error in
kconfig.py, which is made easier by making the warnings available in a
list in Kconfiglib.

It's now configurable whether warnings are printed to stderr or not. In
this case, it still makes sense to print them to stderr as well.

Suggested by Marti Bolivar.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-05-01 20:53:40 +02:00
Ulf Magnusson
73549ad852 scripts: kconfig: Add a Python menuconfig implementation
This commit adds a Kconfiglib-based menuconfig implementation, built
with the standard Python 'curses' module. A new 'pymenuconfig' target is
added to run it.

The C tools are kept for now. Removing them separately allows testing of
pymenuconfig alongside the C tools, and keeps changes small and focused.

A feature is planned for later that shows all symbols -- including those
that aren't currently visible -- along with a search and "jump to"
feature. Loading of arbitrary .config files will be supported later as
well (as opposed to always loading .config/KCONFIG_CONFIG). Those
features are all connected implementation-wise.

For Windows, the wheels at
https://www.lfd.uci.edu/~gohlke/pythonlibs/#curses provide the curses
implementation. They use the standard Python curses module
(_cursesmodule.c), linked against PDCurses.

Running 'python -VV' gives the Python version and bitness, to know which
wheel to install. User documentation will be added once the C tools are
removed and the 'pymenuconfig' target is moved over to 'menuconfig'.

The CMake parts are originally by Sebastian Bøe.

Description, taken from the menuconfig.py docstring:

    Overview
    ========

    A curses-based menuconfig implementation. The interface should feel
    familiar to people used to mconf ('make menuconfig').

    Supports the same keys as mconf, and also supports a set of
    keybindings inspired by Vi:

      J/K     : Down/Up
      L       : Enter menu/Toggle item
      H       : Leave menu
      Ctrl-D/U: Page Down/Page Down
      G/End   : Jump to end of list
      g/Home  : Jump to beginning of list

    The mconf feature where pressing a key jumps to a menu entry with
    that character in it in the current menu isn't supported. A search
    feature with a "jump to" function for jumping directly to a
    particular symbol regardless of where it is defined will be added
    later instead.

    Space and Enter are "smart" and try to do what you'd expect for the
    given menu entry.

    Running
    =======

    menuconfig.py can be run either as a standalone executable or by
    calling the menu.menuconfig() function with an existing Kconfig
    instance. The second option is a bit inflexible in that it will
    still load and save .config, etc.

    When run in standalone mode, the top-level Kconfig file to load can
    be passed as a command-line argument. With no argument, it defaults
    to "Kconfig".

    The KCONFIG_CONFIG environment variable specifies the .config file
    to load (if it exists) and save. If KCONFIG_CONFIG is unset,
    ".config" is used.

    $srctree is supported through Kconfiglib.

    Other features
    ==============

      - Seamless terminal resizing

      - No dependencies on *nix, as the 'curses' module is in the Python
        standard library

      - Unicode text entry

      - Improved information screen compared to mconf:

          * Expressions are split up by their top-level &&/|| operands
            to improve readability

          * Undefined symbols in expressions are pointed out

          * Menus and comments have information displays

          * Kconfig definitions are printed

    Limitations
    ===========

      - Python 3 only

        This is mostly due to Python 2 not having curses.get_wch(),
        which is needed for Unicode support.

      - Doesn't work out of the box on Windows

        Has been tested to work with the wheels provided at
        https://www.lfd.uci.edu/~gohlke/pythonlibs/#curses though.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-05-01 20:51:11 +02:00
Ulf Magnusson
d3bfbfeb17 kconfiglib: Update to get choice.direct_dep in
Update Kconfiglib to upstream revision 509e374dfcadb (+ local Zephyr
modifications) to get commit 509e374dfcadb ("Add Choice.direct_dep
field") in. It is used by the upcoming Python menuconfig implementation
when displaying information about choices.

Origin: https://github.com/zephyrproject-rtos/Kconfiglib/tree/zephyr

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-05-01 20:51:11 +02:00
Ulf Magnusson
b742b62b6e kconfiglib: Update to get split_expr() in
Update Kconfiglib to upstream revision 105c835e70a5b (+ local Zephyr
modifications) to get commit 105c835e70a5b ("Add helper for splitting
expressions") in. It will simplify the 'select' logic in genrest.py and
the upcoming menuconfig implementation, and also simplifies some
Kconfiglib internals.

split_expr() will also be helpful if genrest.py ever needs to split
other expressions, e.g. over multiple lines.

Origin: https://github.com/zephyrproject-rtos/Kconfiglib/tree/zephyr

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-04-17 08:02:40 -07:00
Ulf Magnusson
8fc44f2988 kconfiglib: Update to e8408a06c68d8
Update Kconfiglib to upstream revision e8408a06c68d8 (+ local Zephyr
modifications) to get commit e8408a06c68d8 ("Move sanity checking to
after _finalize_tree()") in. It fixes some obscure false-positive
warnings for named choices that came up in
https://github.com/zephyrproject-rtos/zephyr/issues/6948.

Origin: https://github.com/zephyrproject-rtos/Kconfiglib/tree/zephyr

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-04-06 20:53:41 +02:00
Ulf Magnusson
db28a5d8b7 kconfiglib: Update to 981d24aff7654
Update Kconfiglib to upstream revision 981d24aff7654 (+ local Zephyr
modifications) to get commit 981d24aff7654 ("Set is_menuconfig True on
the top menu") in. It will be needed by the menuconfig implementation.

Origin: https://github.com/zephyrproject-rtos/Kconfiglib/tree/zephyr

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-04-05 13:10:12 +02:00
Ulf Magnusson
46a172ae3c kconfiglib: Update to 2259d353426f1
Update Kconfiglib to upstream revision 2259d353426f1 (+ local Zephyr
modifications) to get commit 2259d353426f1 ("Generalize is_menuconfig to
non-symbol items") in. It will be used by the menuconfig implementation.

Origin: https://github.com/zephyrproject-rtos/Kconfiglib/tree/zephyr

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-04-04 15:29:21 -04:00
Ulf Magnusson
4b35875719 kconfiglib: Update to 7245bad9ebb58
Update Kconfiglib to upstream revision 7245bad9ebb58 (+ local Zephyr
modifications), to get the following improvements/fixes in:

 - Print a warning if an int or hex symbol is assigned a value that lies
   outside an active 'range' constraint.

 - Turn 'FOO' into a link in 'select FOO' when generating the Kconfig
   reference documentation.

 - Parenthesize '&&' expressions within '||' expressions --
   (A && B) || (C && D) is more readable than A && B || C && D.

The final two fixes will only be visible once the fix for #5622 gets in.

Fixes #6749
Fixes #6844

Origin: https://github.com/zephyrproject-rtos/Kconfiglib/tree/zephyr

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-03-28 23:56:49 +02:00
Ulf Magnusson
e3892d9351 scripts: kconfig: Give symbol locations in warnings
Warnings from Kconfiglib itself always give symbol locations, but the
custom value mismatch warning in kconfig.py doesn't. Make it print the
symbol location(s) too. This is helpful when debugging.

Before:

    warning: UART_QMSI_0_HW_FC was assigned the value "y" but got the
    value "n" -- check dependencies

After:

    warning: UART_QMSI_0_HW_FC (defined at
    .../drivers/serial/Kconfig.qmsi:35,
    .../arch/x86/soc/intel_quark/quark_se/Kconfig.defconfig.series:194)
    was assigned the value "y" but got the value "n" -- check
    dependencies

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2018-03-26 10:03:57 +02:00