The code relocation feature allows code and data section to be located
inside a designated RAM region.
Currently, this feature supports relocation of code (text) and data
including mutable data (data), read-only data (rodata), and
zero-initialized data (bss).
However, relocation of non-initialized data sections was not previously
supported, meaning that any data annotated with the __noinit attribute
could not be relocated into the desired RAM region.
This patch adds a NOINIT memory-type which can be used implicitly or
explictly in the zephyr_code_relocate() CMake function. This causes the
build system to generate additional linker-script section-matching rules.
By the nature of noinit data, no action is required by Zephyr at boot.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Since compiler options are unconditionally passed to the script,
we may pass arguments that are not recognized. So we change to
only parse known arguments. Currently, it only cares about -l,
which is related to linking libraries.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When not using dynamic interrupt mapping, various interrupt tables are
configured to be stored in read-only memory in the linker script.. Mark
them const so that the linker doesn't complain.
This affects _sw_isr_table, _irq_vector_table, and z_shared_sw_isr_table in
arch/common along with _VectorTable in arch/arc.
Signed-off-by: Keith Packard <keithp@keithp.com>
The string_create_helper function has the arguments: load_address_in_flash
and is_copy. These arguments take logical values, but previously the
calling code used 1 and 0 rather than the more idiomatic True and False.
This patch corrects the issue by replacing use of int values with bool
values.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
This patch makes various simplifications to the code structure which make
it more concise by reducing repetition.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Previously the script used templates where the fields were identified by
numeric ordinal-identified placeholders. This makes the code hard to read
because it is hard to tell which string corresponds to which placeholder
in the substitution.
This patch corrects the issue by replacing the ordinal placeholders with
named placeholders.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
In Python, type annotations can be quoted to avoid forward references.
However, if "from __future__ import annotations" is present, Python will
always evaluate type annotations in a deferred manner, making the quotes
unnecessary.
The ruff python linter produces UP037 warnings to indicate cases where
quoting of type annotations can be removed. This patch corrects the
warnings.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
PEP585 enabled collections in the Python standard library (like tuple) to
be used as generic type annotations directly, instead of importing
analogous members from the typing module (like typing.Tuple).
The ruff python linter produces a UP006 warning if the deprecated type
annotations continue to be used.
This patch corrects the issue by correcting the single instance where
PEP585-style type annotations can be used.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
The ruff python linter produces a SIM401 warning when a dictionary is
accessed using if-statements to check for key presence. In this case, the
dict.get() method could be used instead.
For example:
value = foo["bar"] if "bar" in foo else 0
...can be replaced with:
value = foo.get("bar", 0)
This patch corrects the single instance of this issue in the script.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
The ruff python linter produces a SIM102 warning when there are nested if
statements which can be collapsed into a single if statement:
if foo:
if bar:
...
...becomes...
if foo and bar:
...
This patch corrects the single instance of this issue in the script.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
The ruff python linter produces a I001 warning when the imports of a Python
script are not sorted. This patch corrects the issue by sorting them.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
The script can be made more concise by combining the imports of three
classes from the typing module into a single line.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
The ruff python linter produces a B028 warning when the warnings.warn()
function is called with a stacklevel parameter.
By default the function will set a stacklevel of 1 which causes it to
output the stack frame of the line where the function is called without
any context information from higher up the stack.
It is recommended to use a stacklevel of 2 or higher. Therfore, this patch
sets the stacklevel parameter of all warnings.warn() calls to to 2.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Ruff is the Zephyr projects supported Python formatting tool. This patch
applies auto-formatting gen_relocate_app.py in preparation for coming
tidy-ups and improvements.
With the Ruff auto-formatter applied, error E501 can be removed from
.ruff-excludes.toml exclusion rules.
gen_relocate_app.py has also been removed from the format exclude list.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
As part of relocation, this script matches each source file to its
corresponding object file. This matching is done inside of get_obj_filename
which is called once per source file.
get_obj_filename traverses the entire build directory on every invocation.
This is unnecessary since built object files don't change. On a
sufficiently large project (like mine), this script takes over a minute and
the majority of that time is spent needlessly traversing the build
directory again and again.
Caching the list of object files enables this script to run in less than a
second.
I tested by building my project (which enables the relocation script) and
comparing memory breakdown before / after.
Signed-off-by: Galen Krulce <gkrulce@meta.com>
Tweak gen_app_partitions.py to be able to generate cmake linker
generator output as well as ld-script fragments.
Signed-off-by: Björn Bergman <bjorn.bergman@iar.com>
Added recently introduced optimizations - generation in string literal form
and faster generation for hexadecimal initializer list form - to gzip path
as well.
Signed-off-by: Irfan Ahmad <irfan.ahmad@siemens.com>
One might want to select the symbols to be relocated inside a file or
a library. To do this, one can use the FILTER argument of
zephyr_code_relocate which must contain a regular expression of the
section names to be selected for relocation.
The test_function_in_sram2 test case in
`tests/application_development/code_relocation` has been updated to
verify that only one function `function_in_sram()` is relocated to ram
and that the function `function_not_relocated()` is not being relocated
when using relocation filter.
Signed-off-by: Sylvain Chouleur <sylvain.chouleur@gmail.com>
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
With code relocation directives passed to the gen_relocate_app.py script
using generated file, then each directive can be place on individual
line in the file and thus free up the `|` character as separator.
Furthermore, a multi-line file with each directive on separate line is
also more user-readable, making debugging easier.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Currently, file2hex.py supports conversion of binary data into hexadecimal
character list format only. The generated list can then be used to embed
the binary data by using the list to initialize an array. However, this
approach is highly inefficient for large binary files.
A close but considerably more efficient alternative is to use string
literals composed of hex characters (in escaped form) to initialize the
array, instead of an initializer list. Benchmarking (with GCC and clang)
indicates that compile time and host memory usage by the compiler can be
more than an order of magnitude less with string literal approach compared
to the initializer list form.
The only caveat is that string literals contain the null character as
terminator so where accurate length is required, the correct length must
be specified explicitly while defining the array.
Signed-off-by: Irfan Ahmad <irfan.ahmad@siemens.com>
Read and convert binary data in chunks of 1024 bytes, instead of 8 bytes.
This significantly reduces the conversion time. The improvement increases
with increasing file sizes but saturates to around 60-61% as file size
approaches 64MiB, and beyond.
The existing generated output format of eight byte-values per line is still
preserved.
Signed-off-by: Irfan Ahmad <irfan.ahmad@siemens.com>
The ELF format allows for multiple string and symbol tables with
complex references between them. This is especially evident when
debugging information is included.
This patch fixes the issues that have been identified with multiple
string tables to allow LLEXT to properly parse those files:
* The symbol table used by LLEXT (LLEXT_MEM_SYMTAB) is now chosen
depending on the loaded file type, and other tables are ignored.
This change is also applied to the SLID injection script.
* The LLEXT string table (LLEXT_MEM_SYMTAB) is now correctly identified
by the symbol table reference, instead of picking the first one.
* VMA range checks only make sense for allocated sections.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
Make scripts with an interpreter line executable so that they can be
invoked directly.
Signed-off-by: Sebastian Huber <sebastian.huber@embedded-brains.de>
Mostly a revert of commit b1def7145f ("arch: deprecate `_current`").
This commit was part of PR #80716 whose initial purpose was about providing
an architecture specific optimization for _current. The actual deprecation
was sneaked in later on without proper discussion.
The Zephyr core always used _current before and that was fine. It is quite
prevalent as well and the alternative is proving rather verbose.
Furthermore, as a concept, the "current thread" is not something that is
necessarily architecture specific. Therefore the primary abstraction
should not carry the arch_ prefix.
Hence this revert.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Add UF2 Family ID for Raspberry Pi 2350 and build
UF2 image by default for Pico 2 board
Signed-off-by: Ryan Grachek <grachek@gmail.com>
Signed-off-by: Andrew Featherstone <andrew.featherstone@gmail.com>
The weak syscall symbols generated by gen_syscalls.py are currently
compiled in the LLEXT subsystem library, which is then linked among all
other Zephyr libraries in an unspecified order. This can cause the weak
symbols to override the actual syscall implementations, leading to
undefined behaviour.
To fix this, the currently generated file is split in two elements:
- syscall_exports_llext.c contains the EXPORT_SYMBOL directives for all
syscalls. This part can be compiled with the LLEXT library and linked
among all other Zephyr libraries, and ensures all syscalls symbols
are preserved by the linker.
- syscall_weakdefs_llext.c contains the weak definitions for all syscalls.
This file is compiled in a separate library that is linked last, so
that the weak symbols are only used if no other implementation is
available.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
Some linkers (e.g. the ARC MWDT one) don't recognize wildcards in file
names if they are not enclosed in quotes. Looks like the quotes have
no negative effect on the GNU ld linker, so just do it unconditionally.
Signed-off-by: Ilya Tagunov <Ilya.Tagunov@synopsys.com>
The gperf tool automatically selects the optimal data type for the
asso_values table, depending on MAX_HASH_VALUE. However, there is
a corner case when the tables generated on different stages of the
build process have different data types, causing a link-time error.
Upgrade the data type for the table from unsigned char to unsigned
short to at least exclude this 8-bit to 16-bit transition. There is
another potential issue with the 16-bit to 32-bit transition, but
it seems not very likely to have 65k kernel objects anytime soon.
Signed-off-by: Ilya Tagunov <Ilya.Tagunov@synopsys.com>
Extend the device subsystem enumeration script to produce a CMake
pre-load script.
This allow CMake linker generator scripts to create iterable sections
based on output from device subsystem enumeration.
This ensures that same functionality is available in both ld linker
templates and the linker generator.
Update linker generators to support the use of the device subsystem
enumeration CMake pre-load script.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
The device enumeration feature requires all devices
to place their API implementation in linker sections
by api type. This commit adds a script which uses
the tag __subsystem to identify all existing driver
API types and generate iterable sections for them.
The script is invoked from the top CMakeLists.txt
Signed-off-by: Bjarki Arge Andreasen <baa@trackunit.com>
Co-authored-by: Pieter De Gendt <pieter.degendt@basalte.be>
Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
`_current` is now functionally equals to `arch_curr_thread()`, remove
its usage in-tree and deprecate it instead of removing it outright,
as it has been with us since forever.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Some auto formatters will wrap long lines of code and insert newlines
that are part of function decls outside of arguments. This change strips
out all newlines so syscall typename regex function as expected.
Signed-off-by: Mark Inderhees <markind@meta.com>
the script shows an incorrect file name, I check the wrong file first, then
I find the script did not print the current reading file name.
Fix this to prevent others from wasting their time on this.
The new error message:
```
[1/179] Generating syscalls.json, struct_tags.json
Error decoding zmk/.../altera_msgdma.c (included in zephyr/.../ethernet.c)
```
Signed-off-by: Sa Sasu <i@sasa.su>
When building an LLEXT-enabled kernel, 62b19ef65c added weak aliases
of all syscall implementation functions to a pointer to NULL, with the
assumption that LLEXT would check the required symbols at link time and
fail if any of them were found.
This check, however, is ineffective in the current implementation: the
actual address that is exported is the rather normal-looking location of
the variable containing the NULL pointer. This defeats the NULL symbol
validity checks in llext_link.c and causes the extension to crash at
runtime by jumping to a location containing a few zeroes in read-only
data memory.
This commit makes sure the alias target is actually placed at address 0
using the llext-sections.ld linker fragment, so that undefined syscall
implementations are exported as NULLs and as such properly flagged at
link time.
The test for this functionality is also updated to reflect the change.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
This commit introduces support for an alternate linking method in the
LLEXT subsystem, called "SLID" (short for Symbol Link Identifier),
enabled by the CONFIG_LLEXT_EXPORT_BUILTINS_BY_SLID Kconfig option.
SLID-based linking uses a unique identifier (integer) to identify
exported symbols, instead of using the symbol name as done currently.
This approach provides several benefits:
* linking is faster because the comparison operation to determine
whether we found the correct symbol in the export table is now an
integer compare, instead of a string compare
* binary size is reduced as symbol names can be dropped from the binary
* confidentiality is improved as a side-effect, as symbol names are no
longer present in the binary
Signed-off-by: Mathieu Choplain <mathieu.choplain@st.com>
Employs the same linkonce magic of sw_isr_table to fix the
multiple definition of the symtab variables issue that I
get in my application build that doesn't use `west`.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
`start_addr` is the address of the first symbol, rename it to
`first_addr` instead as it seems more intuitive and relatable
to the comments.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Append new entry to the symtab list only if it has unique
address.
Added a bit more comments and move the debug print to after
the list is sorted.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
The `symtab_find_symbol_name()` is using an adapted binary
search function to get the entry between 2 addresses, we need
to add a dummy entry at the end so that the search function
can remain simple and straightforward without doing
out-of-bound checks:
20 \
|
|
50 x
|
|
90 x
. |
. |
. |
dummy /
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Namespaced the generated headers with `zephyr` to prevent
potential conflict with other headers.
Introduce a temporary Kconfig `LEGACY_GENERATED_INCLUDE_PATH`
that is enabled by default. This allows the developers to
continue the use of the old include paths for the time being
until it is deprecated and eventually removed. The Kconfig will
generate a build-time warning message, similar to the
`CONFIG_TIMER_RANDOM_GENERATOR`.
Updated the includes path of in-tree sources accordingly.
Most of the changes here are scripted, check the PR for more
info.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Use pyelftools to extract the symbol table from the
link stage executable. Then, filter out the function names
and sort them based on their offsets before writing into the
`symtab.c`, this is similar to how the `isr_tables` works.
To access the structure, simply include the new header:
```c
#include <zephyr/debug/symtab.h>
```
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
A new Kconfig option which generates syscall stubs assuming that
extensions will always run on userspace, thus simplifying linking
them, as there's no need for z_impl_ stubs (used for direct syscalls),
CONFIG_LLEXT_EDK_USERSPACE_ONLY.
While defining __ZEPHYR_USER__ could have the same effect for optmised
builds, people building extensions on debug environments - thus
non-optimised - would suffer, as they'd need to somehow make the stubs
available (by either exporting the symbol or implementing dummy stubs).
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Pull in the latest uf2conv.py and newly needed uf2families.json file
to address Python warning on Python 3.12 and keep us in sync.
Signed-off-by: Peter Johanson <peter@peterjohanson.com>
