This moves the k_* memory management functions from sys/ into
kernel/ includes, as there are kernel public APIs. The z_*
functions are further separated into the kernel internal
header directory.
Also made a quick change to doxygen to group sys_mem_* into
the OS Memory Management group so they will appear in doc.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Move the syscall_handler.h header, used internally only to a dedicated
internal folder that should not be used outside of Zephyr.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When Zephyr is compiled using LLVM toolchain, we don't need to link with
libgcc to resolve libc dependencies. With this patch, the trick will be
applied only when the GNU compiler is used. Otherwise, we will just link
libc, which works for LLVM.
Signed-off-by: Patryk Duda <pdk@semihalf.com>
The init infrastructure, found in `init.h`, is currently used by:
- `SYS_INIT`: to call functions before `main`
- `DEVICE_*`: to initialize devices
They are all sorted according to an initialization level + a priority.
`SYS_INIT` calls are really orthogonal to devices, however, the required
function signature requires a `const struct device *dev` as a first
argument. The only reason for that is because the same init machinery is
used by devices, so we have something like:
```c
struct init_entry {
int (*init)(const struct device *dev);
/* only set by DEVICE_*, otherwise NULL */
const struct device *dev;
}
```
As a result, we end up with such weird/ugly pattern:
```c
static int my_init(const struct device *dev)
{
/* always NULL! add ARG_UNUSED to avoid compiler warning */
ARG_UNUSED(dev);
...
}
```
This is really a result of poor internals isolation. This patch proposes
a to make init entries more flexible so that they can accept sytem
initialization calls like this:
```c
static int my_init(void)
{
...
}
```
This is achieved using a union:
```c
union init_function {
/* for SYS_INIT, used when init_entry.dev == NULL */
int (*sys)(void);
/* for DEVICE*, used when init_entry.dev != NULL */
int (*dev)(const struct device *dev);
};
struct init_entry {
/* stores init function (either for SYS_INIT or DEVICE*)
union init_function init_fn;
/* stores device pointer for DEVICE*, NULL for SYS_INIT. Allows
* to know which union entry to call.
*/
const struct device *dev;
}
```
This solution **does not increase ROM usage**, and allows to offer clean
public APIs for both SYS_INIT and DEVICE*. Note that however, init
machinery keeps a coupling with devices.
**NOTE**: This is a breaking change! All `SYS_INIT` functions will need
to be converted to the new signature. See the script offered in the
following commit.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
init: convert SYS_INIT functions to the new signature
Conversion scripted using scripts/utils/migrate_sys_init.py.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
manifest: update projects for SYS_INIT changes
Update modules with updated SYS_INIT calls:
- hal_ti
- lvgl
- sof
- TraceRecorderSource
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
tests: devicetree: devices: adjust test
Adjust test according to the recently introduced SYS_INIT
infrastructure.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
tests: kernel: threads: adjust SYS_INIT call
Adjust to the new signature: int (*init_fn)(void);
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
The `sys/stat.h` header has never been a part of ISO C so move it
to `zephyr/include/posix/sys/`.
To ensure a smooth migration, leave a stub header in
`lib/libc/minimal/include/sys/` that prints a deprecation warning
suggesting developers either include `<zephyr/posix/sys/stat.h>`
or use `CONFIG_POSIX_API=y`.
Signed-off-by: Chris Friedt <cfriedt@meta.com>
This commit relocates the newlib libc configurations under the top-
level libc Kconfig to a new Kconfig under `lib/libc/newlib` for
improved organisation of the libc configurations.
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
This commit updates the Newlib integration to define `_ANSI_SOURCE`
in order to prevent Newlib from defining POSIX primitives in its
headers when GNU dialect is used (`-std=gnu*`).
Newlib `features.h` defines `_DEFAULT_SOURCE` when `__STRICT_ANSI__`
is not defined by GCC (i.e. when `-std=gnu*`), which results in the
Newlib headers defining POSIX primitives that are in conflict with the
POSIX primitives defined by Zephyr.
Newlib must not define POSIX primitives unless the feature test macros
such as `_POSIX_SOURCE`, `_GNU_SOURCE` and `_DEFAULT_SOURCE` are
explicitly defined.
Note that `-std=gnu` does not imply `_GNU_SOURCE` or `_DEFAULT_SOURCE`
because it is only supposed to instruct the compiler to use the GNU C
language dialect (i.e. GNU C language extensions).
Refer to the GitHub issue #52739 for more details.
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
Add build assert to make sure _RETARGETABLE_LOCKING is enabled in
toolchain, When _RETARGETABLE_LOCKING is enabled, "_LOCK_T" is "__lock"
pointer type, otherwise "_LOCK_T" is "int" type, so there will be the
following compile warnings when toolchain doesn't enable
_RETARGETABLE_LOCKING:
zephyr/lib/libc/newlib/libc-hooks.c:416:13: warning: cast to pointer from
integer of different size [-Wint-to-pointer-cast]
416 | k_sem_take((struct k_sem *)lock, K_FOREVER);
| ^
zephyr/lib/libc/newlib/libc-hooks.c: In function '__retarget_lock_acquire
_recursive':
zephyr/lib/libc/newlib/libc-hooks.c:423:15: warning: cast to pointer from
integer of different size [-Wint-to-pointer-cast]
423 | k_mutex_lock((struct k_mutex *)lock, K_FOREVER);
| ^
...
Signed-off-by: Jiafei Pan <Jiafei.Pan@nxp.com>
When CONFIG_POSIX_CLOCK is enabled, we should have implementation
of gettimeofday() and therefore time(NULL) should return correct
time, instead of -1.
Signed-off-by: Seppo Takalo <seppo.takalo@nordicsemi.no>
This commit changes the invocation of the newlib malloc heap
initialisation function such that it is executed during the POST_KERNEL
phase instead of the APPLICATION phase.
This is necessary in order to ensure that the application
initialisation functions (i.e. the functions called during the
APPLICATIION phase) can make use of the libc heap.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
In order to bring consistency in-tree, migrate all lib code to the new
prefix <zephyr/...>. Note that the conversion has been scripted, refer
to zephyrproject-rtos#45388 for more details.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Fix a variable declaration type conflict:
libc-hooks.c:92:16: error: conflicting types for '_heap_sentry'
92 | extern void *_heap_sentry;
soc.h:78:13: note: previous declaration of '_heap_sentry' was here
Fixes#44926
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Calling gettimeofday() from _gettimeofday() in a non-Posix build
environment can result in a recursive call loop, causing a stack
overflow. Modify _gettimeofday() to return -1 for non-posix systems
(the previous behaviour that was added in #22508).
Fixes#41095
Signed-off-by: Binu Jacob <bjj@planetinnovation.com.au>
* add generic heap event listener module that can be used
for notifying an application of heap-related events
* use the listener module in newlib libc hooks
* add a unit test
Signed-off-by: Damian Krolik <damian.krolik@nordicsemi.no>
The commit 9bd1483afeb18f4225ec7b0340b0d4e20efb7d01 was added as a
workaround for the Xtensa initial malloc failure bug.
This bug has been fixed in the Zephyr SDK 0.13.1 release and therefore
this workaround is no longer needed.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit removes the `z_` prefix from the stdio syscall functions
(`z_zephyr_write_stdout` and `z_zephyr_read_stdin`) since it is
redundant and does not align with the convention used by the equivalent
minimal libc syscall functions (e.g. `zephyr_fputc` and
`zephyr_fwrite`).
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
The commit 4344e27c26 changed the syscall
function invocation in the `write()` and `read()` functions to the
direct syscall implementation function invocation by mistake.
The newlib `write()` and `read()` functions must call the
`z_zephyr_write_stdout()` and `z_zephyr_read_stdin()` syscall functions
in order to function properly in a user mode context.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
For the Xtensa platforms (e.g. qemu_xtensa), the first `malloc` call
may fail if the newlib heap base address is such that the first `sbrk`
call returns a 4096-byte aligned address.
Here we add a workaround for Xtensa that allocates and immediately
frees a 16-byte memory block during initialisation so that all
subsequent `malloc` calls succeed.
This commit needs to be reverted once the issue #38258 is fixed.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit adds the newlib retargetable locking interface function
implementations in order to make newlib functions thread safe.
The newlib retargetable locking interface is internally called by the
standard C library functions provided by newlib to synchronise access
to the internal shared resources.
By default, the retargetable locking interface functions defined within
the newlib library are no-op. When multi-threading is enabled (i.e.
`CONFIG_MULTITHREADING=y`), the Zephyr-side retargetable locking
interface implementations override the default newlib implementation
and provide locking mechanism.
The retargetable locking interface may be called with either a static
(`__lock__...`) or a dynamic lock.
The static locks are statically allocated and initialised immediately
after kernel initialisation by `newlib_locks_prepare`.
The dynamic locks are allocated and de-allocated through the
`__retargetable_lock_init[_recursive]` and
`__retarget_lock_close_[recurisve]` functions as necessary by the
newlib functions. These locks are allocated in the newlib heap using
the `malloc` function when userspace is not enabled -- this is safe
because the internal multi-threaded malloc lock implementations
(`__malloc_lock` and `__malloc_unlock`) call the retargetable locking
interface with a static lock (`__lock__malloc_recursive_mutex`). When
userspace is enabled, the dynamic locks are allocated and freed through
`k_object_alloc` and `k_object_release`.
Note that the lock implementations used here are `k_mutex` and `k_sem`
instead of `sys_mutex` and `sys_sem` because the Zephyr kernel does not
currently support dynamic allocation of the latter. These locks should
be updated to use `sys_mutex` and `sys_sem` when the Zephyr becomes
capable of dynamically allocating them in the future.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit adds a new `CONFIG_NEWLIB_MIN_REQUIRED_HEAP_SIZE` config
that allows user to specify the minimum required heap size for the
newlib heap, and makes `malloc_prepare` validate that the memory space
available for the newlib heap is greater than this value.
The default minimum required heap size values were empiricially
determined, so as to allow the basic standard C functions such as
`printf` and `scanf` to work properly.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
The time() function works correctly with the minimal libc, but always
returns -1 with the newlib libc. This is due to the _gettimeofday hook
being implemented that way.
Fix that by calling gettimeofday in the _gettimeofday hook instead of
returning -1.
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
This commit removes the lock inside the newlib internal `_sbrk`
function, which is called by `malloc` when additional heap memory is
needed.
This lock is no longer required because any calls to the `malloc`
function are synchronised by the `__malloc_lock` and `__malloc_unlock`
functions.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit adds a lock implementation for the newlib heap memory
management functions (`malloc` and `free`).
The `__malloc_lock` and `__malloc_unlock` functions are called by the
newlib `malloc` and `free` functions to synchronise access to the heap
region.
Without this lock, making use of the `malloc` and `free` functions from
multiple threads will result in the corruption of the heap region.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Split ARM and ARM64 architectures.
Details:
- CONFIG_ARM64 is decoupled from CONFIG_ARM (not a subset anymore)
- Arch and include AArch64 files are in a dedicated directory
(arch/arm64 and include/arch/arm64)
- AArch64 boards and SoC are moved to soc/arm64 and boards/arm64
- AArch64-specific DTS files are moved to dts/arm64
- The A72 support for the bcm_vk/viper board is moved in the
boards/bcm_vk/viper directory
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Previously, newlib claimed all free physical memory in the
system.
Now, the kernel manages this, allowing for memory to be
used via k_mem_map() calls.
Establish an upper bound to how much newlib will try to
claim on system startup, instead of trying to take all
of it, allowing other parts of the system to also map
anonymous memory.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
We now draw heap memory from an anonymous memory mapping
instead of a hard-coded region past the kernel image,
which is no longer mapped by default.
Some readability cleanups were made to a particuarly
horrible set of nested ifdefs. A few types were adjusted.
sbrk()'s count argument is an intptr_t, not an int.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Previously, newlib claimed all free physical memory in the
system.
Now, the kernel manages this, allowing for memory to be
used via k_mem_map() calls.
Establish an upper bound to how much newlib will try to
claim on system startup, instead of trying to take all
of it, allowing other parts of the system to also map
anonymous memory.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
We now draw heap memory from an anonymous memory mapping
instead of a hard-coded region past the kernel image,
which is no longer mapped by default.
Some readability cleanups were made to a particuarly
horrible set of nested ifdefs. A few types were adjusted.
sbrk()'s count argument is an intptr_t, not an int.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Fixes: #28650
Linking with newlib now defines the following linker flags as:
```
${CMAKE_C_LINKER_WRAPPER_FLAG}${CMAKE_LINK_LIBRARY_FLAG}c
${CMAKE_C_LINKER_WRAPPER_FLAG}${CMAKE_LINK_LIBRARY_FLAG}gcc
c
```
This is needed because when linking with newlib on aarch64, then libgcc
has a link dependency to libc (strchr), but libc also has dependencies
to libgcc.
CMake is capable of handling circular link dependencies for CMake
defined static libraries, which can be further controlled using
`LINK_INTERFACE_MULTIPLICITY`.
However, libc and libgcc are not regular CMake libraries, and is seen as
linker flags by CMake, and thus symbol de-duplications will be
performed.
CMake link options cannot be used, as that will place those libs first
on the linker invocation. -Wl,--start-group is problematic as the
placement of -lc and -lgcc is not guaranteed in case later libraries are
also using -lc / -libbgcc as interface linker flags.
Thus, we resort to use
`${CMAKE_C_LINKER_WRAPPER_FLAG}${CMAKE_LINK_LIBRARY_FLAG}`
as this ensures the uniqueness and thus avoids symbol de-duplication
which means libc will be followed by libgcc, which is finally followed
by libc again.
It would have been possible to use `-lc` directly, but there is a risk
that an externally library is also adding `-lc` and thus de-duplication
and re-arrangement of this flag happens. This risk is in theory also
existing with this fix, but the long nature of this link flag with using
`${CMAKE_C_LINKER_WRAPPER_FLAG}` would likely indicate a similar fix and
thus those libraries will stay in order.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Default buffer mode is setup by fopen depending on isatty value.
Expect only 0, 1 & 2 to be tty for CONFIG_POSIX_API cases.
This way regular files are opened in block buffering mode (instead
of line buffering mode) which really speed up fread/fwrite
operations on FS.
Signed-off-by: Arnaud Mouiche <arnaud.mouiche@invoxia.com>
Now that device_api attribute is unmodified at runtime, as well as all
the other attributes, it is possible to switch all device driver
instance to be constant.
A coccinelle rule is used for this:
@r_const_dev_1
disable optional_qualifier
@
@@
-struct device *
+const struct device *
@r_const_dev_2
disable optional_qualifier
@
@@
-struct device * const
+const struct device *
Fixes#27399
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
Suppress the coverity warning on using the semaphore as
this semaphore is used and freed only in this function.
Fixes: #18960
Signed-off-by: David Leach <david.leach@nxp.com>
This whole code block is ifdef'ed around
CONFIG_NEWLIB_LIBC_ALIGNED_HEAP_SIZE being NOT defined,
remove as this can never be true.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
We've come a long way since this was written to implement
generic ram bounds definitions and MPU capabilities,
use them here.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The version as shipped in Newlib itself is coded a bit sloppily for an
embedded environment. We thus want to override it (and make it weak, to
allow user apps to override it in turn, if needed). The desired
properties of the implementation are:
1. It should call _write() (Newlib implementation calls write()).
2. It should be minimal (Newlib implementation allocates message
on the stack, i.e. misses "static const").
Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
Move defines for _RAM_ADDR, _RAM_SIZE, _ROM_ADDR, and _ROM_ADDR into
the linker.ld and thus remove dts_fixup.h. We rework to use
DT_REG_ADDR and DT_REG_SIZE on DT_CHOSEN(zephyr_sram) and
DT_CHOSEN(zephyr_flash).
Also fixup use of _RAM_ADDR/_RAM_SIZE in newlib/libc-hooks.c.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Replace DT_PHYS_RAM_ADDR and DT_RAM_SIZE with DT_REG_ADDR/DT_REG_SIZE
for the DT_CHOSEN(zephyr_sram) node.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
With the change in SDK 0.11.1 to newlib to remove
-DMISSING_SYSCALL_NAMES we now need to implement a version of
_gettimeofday. Previously with pre SDK 0.11.1 we had a recursive mess
of _gettimeofday_r -> gettimeofday -> _gettimeofday_r. (which are all
implemented in newlib and thus we didn't get a link error).
With SDK 0.11.1 we have: _gettimeofday_r -> _gettimeofday. And we
should provide a version of _gettimeofday.
Fixes#22484
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
On xtensa we always need to implement the reentrant fs syscall
functions. So remove the #ifndef CONFIG_POSIX_API protection around
them and add needed externs.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
The xcc specific reentrant syscall implementations are actually useful
for xtensa in general. So move that code from being specific to
intel_s1000 / xcc into generic newlib/libc-hooks.c. This is in prep
for the Zephyr SDK dropping -DMISSING_SYSCALL_NAMES which will make
its version of newlib on xtensa match behavior with xcc.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
This commit refactors kernel and arch headers to establish a boundary
between private and public interface headers.
The refactoring strategy used in this commit is detailed in the issue
This commit introduces the following major changes:
1. Establish a clear boundary between private and public headers by
removing "kernel/include" and "arch/*/include" from the global
include paths. Ideally, only kernel/ and arch/*/ source files should
reference the headers in these directories. If these headers must be
used by a component, these include paths shall be manually added to
the CMakeLists.txt file of the component. This is intended to
discourage applications from including private kernel and arch
headers either knowingly and unknowingly.
- kernel/include/ (PRIVATE)
This directory contains the private headers that provide private
kernel definitions which should not be visible outside the kernel
and arch source code. All public kernel definitions must be added
to an appropriate header located under include/.
- arch/*/include/ (PRIVATE)
This directory contains the private headers that provide private
architecture-specific definitions which should not be visible
outside the arch and kernel source code. All public architecture-
specific definitions must be added to an appropriate header located
under include/arch/*/.
- include/ AND include/sys/ (PUBLIC)
This directory contains the public headers that provide public
kernel definitions which can be referenced by both kernel and
application code.
- include/arch/*/ (PUBLIC)
This directory contains the public headers that provide public
architecture-specific definitions which can be referenced by both
kernel and application code.
2. Split arch_interface.h into "kernel-to-arch interface" and "public
arch interface" divisions.
- kernel/include/kernel_arch_interface.h
* provides private "kernel-to-arch interface" definition.
* includes arch/*/include/kernel_arch_func.h to ensure that the
interface function implementations are always available.
* includes sys/arch_interface.h so that public arch interface
definitions are automatically included when including this file.
- arch/*/include/kernel_arch_func.h
* provides architecture-specific "kernel-to-arch interface"
implementation.
* only the functions that will be used in kernel and arch source
files are defined here.
- include/sys/arch_interface.h
* provides "public arch interface" definition.
* includes include/arch/arch_inlines.h to ensure that the
architecture-specific public inline interface function
implementations are always available.
- include/arch/arch_inlines.h
* includes architecture-specific arch_inlines.h in
include/arch/*/arch_inline.h.
- include/arch/*/arch_inline.h
* provides architecture-specific "public arch interface" inline
function implementation.
* supersedes include/sys/arch_inline.h.
3. Refactor kernel and the existing architecture implementations.
- Remove circular dependency of kernel and arch headers. The
following general rules should be observed:
* Never include any private headers from public headers
* Never include kernel_internal.h in kernel_arch_data.h
* Always include kernel_arch_data.h from kernel_arch_func.h
* Never include kernel.h from kernel_struct.h either directly or
indirectly. Only add the kernel structures that must be referenced
from public arch headers in this file.
- Relocate syscall_handler.h to include/ so it can be used in the
public code. This is necessary because many user-mode public codes
reference the functions defined in this header.
- Relocate kernel_arch_thread.h to include/arch/*/thread.h. This is
necessary to provide architecture-specific thread definition for
'struct k_thread' in kernel.h.
- Remove any private header dependencies from public headers using
the following methods:
* If dependency is not required, simply omit
* If dependency is required,
- Relocate a portion of the required dependencies from the
private header to an appropriate public header OR
- Relocate the required private header to make it public.
This commit supersedes #20047, addresses #19666, and fixes#3056.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
These calls are buildable on common sanitycheck platforms, but are not
invoked at runtime in any tests accessible to CI. The changes are
mostly mechanical, so the risk is low, but this commit is separated
from the main API change to allow for more careful review.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
