The string_create_helper function has the arguments: load_address_in_flash
and is_copy. These arguments take logical values, but previously the
calling code used 1 and 0 rather than the more idiomatic True and False.
This patch corrects the issue by replacing use of int values with bool
values.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
This patch makes various simplifications to the code structure which make
it more concise by reducing repetition.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Previously the script used templates where the fields were identified by
numeric ordinal-identified placeholders. This makes the code hard to read
because it is hard to tell which string corresponds to which placeholder
in the substitution.
This patch corrects the issue by replacing the ordinal placeholders with
named placeholders.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
In Python, type annotations can be quoted to avoid forward references.
However, if "from __future__ import annotations" is present, Python will
always evaluate type annotations in a deferred manner, making the quotes
unnecessary.
The ruff python linter produces UP037 warnings to indicate cases where
quoting of type annotations can be removed. This patch corrects the
warnings.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
PEP585 enabled collections in the Python standard library (like tuple) to
be used as generic type annotations directly, instead of importing
analogous members from the typing module (like typing.Tuple).
The ruff python linter produces a UP006 warning if the deprecated type
annotations continue to be used.
This patch corrects the issue by correcting the single instance where
PEP585-style type annotations can be used.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
The ruff python linter produces a SIM401 warning when a dictionary is
accessed using if-statements to check for key presence. In this case, the
dict.get() method could be used instead.
For example:
value = foo["bar"] if "bar" in foo else 0
...can be replaced with:
value = foo.get("bar", 0)
This patch corrects the single instance of this issue in the script.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
The ruff python linter produces a SIM102 warning when there are nested if
statements which can be collapsed into a single if statement:
if foo:
if bar:
...
...becomes...
if foo and bar:
...
This patch corrects the single instance of this issue in the script.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
The ruff python linter produces a I001 warning when the imports of a Python
script are not sorted. This patch corrects the issue by sorting them.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
The script can be made more concise by combining the imports of three
classes from the typing module into a single line.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
The ruff python linter produces a B028 warning when the warnings.warn()
function is called with a stacklevel parameter.
By default the function will set a stacklevel of 1 which causes it to
output the stack frame of the line where the function is called without
any context information from higher up the stack.
It is recommended to use a stacklevel of 2 or higher. Therfore, this patch
sets the stacklevel parameter of all warnings.warn() calls to to 2.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Ruff is the Zephyr projects supported Python formatting tool. This patch
applies auto-formatting gen_relocate_app.py in preparation for coming
tidy-ups and improvements.
With the Ruff auto-formatter applied, error E501 can be removed from
.ruff-excludes.toml exclusion rules.
gen_relocate_app.py has also been removed from the format exclude list.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
As part of relocation, this script matches each source file to its
corresponding object file. This matching is done inside of get_obj_filename
which is called once per source file.
get_obj_filename traverses the entire build directory on every invocation.
This is unnecessary since built object files don't change. On a
sufficiently large project (like mine), this script takes over a minute and
the majority of that time is spent needlessly traversing the build
directory again and again.
Caching the list of object files enables this script to run in less than a
second.
I tested by building my project (which enables the relocation script) and
comparing memory breakdown before / after.
Signed-off-by: Galen Krulce <gkrulce@meta.com>
One might want to select the symbols to be relocated inside a file or
a library. To do this, one can use the FILTER argument of
zephyr_code_relocate which must contain a regular expression of the
section names to be selected for relocation.
The test_function_in_sram2 test case in
`tests/application_development/code_relocation` has been updated to
verify that only one function `function_in_sram()` is relocated to ram
and that the function `function_not_relocated()` is not being relocated
when using relocation filter.
Signed-off-by: Sylvain Chouleur <sylvain.chouleur@gmail.com>
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
With code relocation directives passed to the gen_relocate_app.py script
using generated file, then each directive can be place on individual
line in the file and thus free up the `|` character as separator.
Furthermore, a multi-line file with each directive on separate line is
also more user-readable, making debugging easier.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Make scripts with an interpreter line executable so that they can be
invoked directly.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
When using the code and data relocation feature, every relocated symbol
would be marked with `KEEP()` in the generated linker script. Therefore,
if any input files contained unused code, then it wouldn't be discarded
by the linker, even when invoked with `--gc-sections`.
This can cause unexpected bloat, or other link-time issues stemming from
some symbols being discarded and others not.
On the other hand, this behavior has been present since the feature's
introduction, so it should remain default for the users who rely on it.
This patch introduces support for `zephyr_code_relocate(... NOKEEP)`.
This will suppress the generation of `KEEP()` statements for all symbols
in a particular library or set of files.
Much like `NOCOPY`, the `NOKEEP` flag is passed to `gen_relocate_app.py`
in string form. The script is now equipped to handle multiple such flags
when passed from CMake as a semicolon-separated list, like so:
"SRAM2:NOCOPY;NOKEEP:/path/to/file1.c;/path/to/file2.c"
Documentation and tests are updated here as well.
Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
For applications relocating big parts of the code with many sections,
builds were failing on Windows due to hitting the max command-line
length on that platform.
Fix this by using a file to store the dictionary passed to the python
script.
Fixes https://github.com/zephyrproject-rtos/zephyr/issues/60994.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Use the "device_deps" naming scheme to emphasize we are storing device
dependencies. The fact we are using device handles to store them is an
implementation detail.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
The code_relocation feature creates generic section names that sometimes
conflict with already existing names.
This patch adds a '_reloc_' word to the created names to reduce the risk
of conflict.
This solves #54785.
Signed-off-by: Björn Stenberg <bjorn@haxx.se>
This refactors the script to introduce a SectionKind enum that
represents each of the kinds of sections handled by this system (text,
data, rodata or bss), using that to improve the code structure by
reducing the use of strings in favor of indicating the use of values by
their types.
Signed-off-by: Peter Marheine <pmarheine@chromium.org>
When using code_data_relocation the script identifies sections that
should be relocated, but cannot guarantee that section names are unique
across all linked files. If a section name is not unique then matching
all sections with that name across all input files will relocate more
data than intended.
As a simple example, if both file1 and file2 contain .rodata.strings
sections, then if gen_relocate_app inspects only file1 it will generate
a linker script fragment `*(.rodata.strings)` that matches both object
files.
This commit changes gen_relocate_app to make the linker match on object
files as well (`*file1.o(.rodata.strings)`) to ensure unwanted sections
don't get relocated.
Signed-off-by: Peter Marheine <pmarheine@chromium.org>
Disables allowing the python argparse library from automatically
shortening command line arguments, this prevents issues whereby
a new command is added and code that wrongly uses the shortened
command of an existing argument which is the same as the new
command being added will silently change script behaviour.
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
Update gen_relocate_app.py to use "|" to separate code relocation
directives, and ";" to separate multiple files in a relocation directive.
This will enable multiple files to be passed to zephyr_code_relocate,
as well as multiple files to be passed from a CMake generator expression.
The script will then seperate these files and relocate each according to
the arguments given to zephyr_code_relocate.
Note! This commit will break support for zephyr_code_relocate until
the CMake function is updated
Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
This is a follow-up to commit b1a3ce4016.
`parse_input_string()` needs to use `:NOCOPY:`, not `:NOCOPY`, when
partitioning input lines, otherwise, when a line contains the NOCOPY
flag, the file name returned by the function starts with `:` and the
file cannot be then found by the script.
Such problem can be observed in the code_relocation_nocopy sample,
which without this fix does not actually relocate code from ext_code.c.
Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
Some linker scripts also associate memory regions with program headers
(PHDRS [1]). This patch adds support on gen_relocate_app.py to also
place a phdr alongside the memory region.
To keep things simple (and more natural), the format is basically the
same used in linker scripts - an space followed by ':<phdr_name>', like:
SRAM2\ :phdr0:COPY:/home/xyz/zephyr/samples/hello_world/src/main.c
(Note the escape char before the space.)
[1] https://sourceware.org/binutils/docs/ld/PHDRS.html
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
As of today <zephyr/zephyr.h> is 100% equivalent to <zephyr/kernel.h>.
This patch proposes to then include <zephyr/kernel.h> instead of
<zephyr/zephyr.h> since it is more clear that you are including the
Kernel APIs and (probably) nothing else. <zephyr/zephyr.h> sounds like a
catch-all header that may be confusing. Most applications need to
include a bunch of other things to compile, e.g. driver headers or
subsystem headers like BT, logging, etc.
The idea of a catch-all header in Zephyr is probably not feasible
anyway. Reason is that Zephyr is not a library, like it could be for
example `libpython`. Zephyr provides many utilities nowadays: a kernel,
drivers, subsystems, etc and things will likely grow. A catch-all header
would be massive, difficult to keep up-to-date. It is also likely that
an application will only build a small subset. Note that subsystem-level
headers may use a catch-all approach to make things easier, though.
NOTE: This patch is **NOT** removing the header, just removing its usage
in-tree. I'd advocate for its deprecation (add a #warning on it), but I
understand many people will have concerns.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
This implements support for relocating code to chosen memory regions via
the `zephyr_code_relocate` CMake function for RISC-V SoCs. ARM-specific
assumptions that were made by gen_relocate_app.py need to be corrected,
in particular not assuming any particular name for the default RAM
section (which is 'SRAM' for most ARM pltaforms) and not assuming 32-bit
pointers (so the test works on RV64).
Signed-off-by: Peter Marheine <pmarheine@chromium.org>
Most ARM platforms name their ROM region `FLASH`, but this assumption is
not portable. Have gen_relocate_app refer to ROMABLE_REGION instead of
hard-coding the platform-specific memory region name.
Signed-off-by: Peter Marheine <pmarheine@chromium.org>
Only the elftools SymbolTableSection section type provides an
iter_symbols() method, and compilers are free to emit sections that
include the substring '.symtab' in their name which causes errors if
gen_relocate_app examines only the section name. Instead check whether a
section is a symbol table to skip attempting to inspect sections that
are not actually symbol tables.
Signed-off-by: Peter Marheine <pmarheine@chromium.org>
Move scripts needed by the build system and not designed to be run
individually or standalone into the build subfolder.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
