Make sure to provide full paths when saving the intermediate files in
the YAML export, to prevent them from being saved in the source tree.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
file(GENERATE ...) does not update the output file if the content is
unchanged. Since the metadata in build_info.yml mostly depends on the
build configuration, the timestamp of the intermediate file does not get
updated on most rebuilds, while the final file does, due to immediate
file(WRITE ...) calls. Since the latter is newer, no post-process step
is executed and the file is left with commented genexes.
Touching the intermediate file ensures that the post-process step is
performed every time, even if the content is unchanged, restoring the
expected behavior.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
Switching the intermediate file format from JSON to YAML has a very
significant benefit: the file is now loaded via yaml_load(), which
internally calls Python to parse the file into the JSON format that
CMake expects. This means that the file contents are now automatically
escaped properly for JSON; it is a huge improvement over the previous
implementation, which was escaping everything given as input to
to_yaml().
With the removal of the now-redundant call in to_yaml(), escaping is
applied exactly once per value or list, when it is passed to yaml_set().
This allows to convert the logic in zephyr_string(ESCAPE ...) to a more
robust "escape everything" approach.
These changes fix the handling of strings with backslashes and different
types of quotes passed either directly or via generator expression. The
existing tests are updated to cover these cases.
Two other small changes are made in this commit:
- a small check in internal_yaml_list_append() is removed, as the same
issue is already detected by the caller yaml_set() logic.
- the to_yaml() function is modified to initialize the YAML output
variable at the top level, which is the expected behavior. This
resulted in genex temp files sometimes having duplicate lines.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
The intermediate file used to expand the generator expressions is now in
YAML format. This allows for a more robust handling of the data, as the
single quoted strings are a lot easier to escape: inside them every
character is a literal except for two single quotes (which intuitively
map to a literal single quote).
When saving a simple YAML file without genexes, the single quotes in
strings need to be escaped. However, doing so when saving the
intermediate file would make it harder to properly escape them later,
since it could be possible that the expansion of the generator
expressions would introduce new single quotes. For this reason, when
genexes are enabled, the escaping is now done in a single pass inside
the yaml-filter.cmake script.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
Store strings in YAML as single-quoted entries to avoid issues with
special characters. This also fixes a quirk with the current test
suite where the quotes in the expected value are filtered out by the
YAML import.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
To move away from the JSON/YAML terminology, rename the variables that
refer to the intermediate files to "expanded" and "output" files.
Also, clean up a few "C-style" function calls and a typo in a variable
name in the same files.
No functional change introduced in this commit.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
Escape the json input string given to `to_yaml()` function and the
content given to `yaml_set()` function.
This ensures that a string like `foo\bar` becomes `foo\\bar` during
internal CMake json processing and when written to the file it becomes
`foo\bar`.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This commit introduce support for maps in a yaml list.
The yaml_set() function has been extended with the following signature:
> yaml_set(NAME <name> KEY <key>...
> [APPEND] LIST MAP <map1> MAP <map2> MAP ...
> )
where a `MAP <map>` has the form:
`MAP "<key1>: <value1>, <key2>: <value2>, ...`
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This commit adds support for generator expressions in values and lists
to the yaml module.
Generator expressions can only be expanded by CMake after all
configuration code has been executed and the final values of the project
properties are defined. This means that contexts that contain generator
expressions are written twice:
- immediately, during the 'yaml_save()' call, a comment with the raw
unexpanded string is saved instead of the key that uses generator
expressions in the YAML file;
- after the configuration step, a custom command updates the YAML file
contents with the fully expanded values.
This two-step process also allows to overcome the issue of lists that
are extracted from generator expressions, whose elements would be
expanded into a single string if written directly to the YAML file.
Instead, the lists are stored in their CMake string format with a
special marker, expanded by CMake into a temporary JSON file, and the
conversion to a proper list is performed during the build step.
If the saved YAML file for context <name> is needed by further build
steps in this project, the target '<name>_yaml_saved' must be added as a
dependency to ensure the final contents are ready.
Note that when generator expressions are used in the context, the GENEX
keyword must be provided to yaml_set(). This is necessary to avoid
storing the genexes as raw strings in the YAML.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
This introductory commit refactors the `yaml_set` function separating
the bodies of list operations into internal functions while preserving
their original behavior.
The conditions that cause the value to be interpreted as a list are also
verified only once, avoiding multiple checks.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
This reverts commit f24f5288f3.
as it's part of a series of commits causing issues on Windows
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
This reverts commit 8f48758836.
as it's part of a series of commits causing issues on Windows
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
This commit introduce support for maps in a yaml list.
The yaml_set() function has been extended with the following signature:
> yaml_set(NAME <name> KEY <key>...
> [APPEND] LIST MAP <map1> MAP <map2> MAP ...
> )
where a `MAP <map>` has the form:
`MAP "<key1>: <value1>, <key2>: <value2>, ...`
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Targets are not available in script mode.
To support the Zephyr scoping feature used by snippets and yaml module
then this commit moves from using custom targets to use GLOBAL
properties for scopes.
A scope property is prefixed with `<scope>:<property>` to avoid naming
collisions.
A `scope:<scope-name>` global property is used to track created scopes.
Tracking valid scopes ensure that properties are only set on known
scopes and thus catches typos / naming errors.
Add zephyr_scope_exists() and zephyr_get_scoped() to abstract the
implementation details of the scoped property retrieval and refactor
current code to use them.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
Initial CMake yaml module to facilitate reading yaml files into CMake,
update yaml settings and write it back to a file.
The yaml module also supports creation of yaml files from scratch.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
