Commit Graph

207 Commits

Author SHA1 Message Date
Andrew Boie
d5b8fe16ad kernel: add app-facing demand paging APIs
Routines to evict memory, page-in memory, and set pinned state
at runtime.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
9111ec2c19 mmu: add k_mem_free_get()
Return the amount of physical anonymous memory remaining.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
69d39af5e6 kernel: add k_mem_map() interface
Allows applications to increase the data space available to Zephyr
via anonymous memory mappings. Loosely based on mmap().

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
2ca5fb7e06 kernel: add page frame management
Initialize the page frame ontology at boot and update it
when we do memory mappings.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Peter Bigot
7b3dc4856a doc: add a section for timeutil APIs
Describe the role of these APIs, key concepts that they depend on, and
expose the low-level API.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2021-01-20 16:38:56 -05:00
Peter A. Bigot
f6d7595d4e lib/timeutil: add utilities to manage local/reference clock alignment
Provide data structures to capture a timestamp in two different
clocks, monitor the drift between those clocks, and using a base
instant with estimated drift convert between the clocks.

This provides the core technology to convert between system uptime and
an external continuous time scale like TAI (UTC without applying leap
seconds).

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2021-01-20 16:38:56 -05:00
Carlo Caione
e77c841023 cache: Expand the APIs for cache flushing
The only two supported operations for data caches in the cache framework
are currently arch_dcache_flush() and arch_dcache_invd().

This is quite restrictive because for some architectures we also want to
control i-cache and in general we want a finer control over what can be
flushed, invalidated or cleaned. To address these needs this patch
expands the set of operations that can be performed on data and
instruction caches, adding hooks for the operations on the whole cache,
a specific level or a specific address range.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2021-01-19 14:31:02 -05:00
Carlo Caione
20f59c8f1e cache: Rename CACHE_FLUSHING to CACHE_MANAGEMENT
The new APIs are not only dealing with cache flushing. Rename the
Kconfig symbol to CACHE_MANAGEMENT to better reflect this change.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2021-01-19 14:31:02 -05:00
Andy Ross
d2eadfa162 lib/os: P4 Work Queue: Pooled Parallel Preemptible Priority-based
This adds a somewhat special purpose IPC mechanism.  It's intended for
applications which have a "work queue" like architecture of discrete
callback items, but which need the ability to schedule those items
independently in separate threads across multiple CPUs.  So P4 Work
items:

1. Can run at any Zephyr scheduler priority and with any deadline
   (this feature assumes EDF scheduling is enabled)

2. Can be submitted at any time and from any context, including being
   resubmitted from within their own handler.

3. Will preempt any lower priority work as soon as they are runnable,
   according to the standard rules of Zephyr priority scheduling.

4. Run from a pool of worker threads that can be allocated efficiently
   (i.e. you need as many as the number of CPUs plus the number of
   preempted in-progress items, but no more).

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-01-15 11:35:50 -05:00
Daniel Leung
fe477ea6d3 kernel: userspace: aligned memory allocation for dynamic objects
This allows allocating dynamic kernel objects with memory alignment
requirements. The first candidate is for thread objects where,
on some architectures, it must be aligned for saving/restoring
registers.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-01-13 09:43:55 -08:00
Peter Bigot
d12a99588b lib: cbprintf: remove cbprintf_arglen
This function was designed to support the logging infrastructure's
need to copy values from va_list structures.  It did not meet that
need, since some values need to be changed based on additional data
that is only available when the complete format specification is
examined.  Remove the function as unnecessary.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2021-01-07 14:02:06 +01:00
Fabio Utzig
e64f4f950e doc: add kernel stack macros to stack apis
Fix broken references to K_THREAD_STACK_* and K_KERNEL_STACK_* macros
by adding those to the existing stack_apis group.

Signed-off-by: Fabio Utzig <fabio.utzig@nordicsemi.no>
2020-12-18 12:18:50 +01:00
Fabio Utzig
8713da70a3 doc: fix @param usage in callbacks
The proper usage of @param in callbacks is currently unsupported by
Doxygen so not warnings are generated. The issues fixed with this commit
where found while adding support to Doxygen for validating @param in
callbacks like it currently does for functions.

Signed-off-by: Fabio Utzig <fabio.utzig@nordicsemi.no>
2020-12-18 12:18:28 +01:00
Andrew Boie
d2ad783a97 mmu: rename z_mem_map to z_phys_map
Renamed to make its semantics clearer; this function maps
*physical* memory addresses and is not equivalent to
posix mmap(), which might confuse people.

mem_map test case remains the same name as other memory
mapping scenarios will be added in the fullness of time.

Parameter names to z_phys_map adjusted slightly to be more
consistent with names used in other memory mapping functions.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-12-16 08:55:55 -05:00
Alexander Wachter
2331d10d4b include: sys: thread_stack add missing extern "C" on C++
Add C linkage for compiling with C++

Signed-off-by: Alexander Wachter <alexander.wachter@leica-geosystems.com>
2020-12-15 11:13:41 +01:00
Anas Nashif
e0f3833bf7 power: remove SYS_ and sys_ prefixes
Remove SYS_ and sys_ from all PM related functions and defines.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-09 15:18:29 -05:00
Tomasz Bursztyka
e137e9b468 sys_io: Expand the API to 64bits read/write functions
And implement the support for intel64 which is basically the
architecture that will require it for now.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2020-12-08 09:29:20 -05:00
Tomasz Bursztyka
4199cd38f1 drivers/pcie: Add support for MSI multi-message
This enables software MSI "multi-vector" feature, letting the user to
register an isr handler per-MSI message.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2020-12-08 09:29:20 -05:00
Andy Ross
0c15627cc1 lib: Remove sys_mem_pool implementation
This has been replaced by sys_heap now and all dependencies are gone.
Remove.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2020-12-07 21:50:14 -05:00
Andy Ross
40c1b55cc2 lib/os/heap: Add sys_heap_realloc()
Add an optimized realloc() implementation that can successfully expand
allocations in place if there exists enough free memory after the
supplied block.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2020-12-07 21:50:14 -05:00
Peter Bigot
bb99422c8a lib/os: replace z_vprintk with cbprintf
Using the same implementation as the rest of Zephyr reduces code size.

Update options and expected results for formatting test.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2020-11-13 06:38:01 -05:00
Peter Bigot
33103828dc lib: add cbprintf capability
This commit adds a C99 stdio value formatter capability where
generated text is emitted through a callback.  This allows generation
of arbitrarily long output without a buffer, functionality that is
core to printk, logging, and other system and application needs.

The formatter supports most C99 specifications, excluding:
* %Lf long double conversion
* wide character output

Kconfig options allow disabling features like floating-point
conversion if they are not necessary.  By default most conversions are
enabled.

The original z_vprintk() implementation is adapted to meet the
interface requirements of cbvprintf, and made available as an opt-in
feature for space-constrained applications that do not need full
formatting support.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2020-11-13 06:38:01 -05:00
Peter Bigot
ca818d5fb9 sys: util: add BIT64_MASK
BIT/_MASK doesn't necessarily handle 64 bits, so provide comparable
functionality for BIT64/_MASK.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2020-11-13 06:38:01 -05:00
Daniel Leung
9be37553ee timing: do not repeatedly do init()/start()/stop()
We should not be initializing/starting/stoping timing functions
multiple times. So this changes how the timing functions are
structured to allow only one initialization, only start when
stopped, and only stop when started.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-11-11 23:55:49 -05:00
Krzysztof Chruscinski
243a470810 lib: os: ring_buffer: Cleaning checkpatch warnings
Removing line exceeding warnings which appeared after u32_t -> uint32_t
converstion.

Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
2020-11-11 13:17:49 +01:00
Krzysztof Chruscinski
1e46bb3bb5 lib: os: ring_buffer: Allow using full buffer capacity
Previously, ring buffer had capacity of provided buffer size - 1. This
trick was used to distinguish between empty and full states. It had one
drawback: ring buffer could not be used as a pool of equal sized buffers
(using ring_buf_put_claim and ring_buf_get_claim).
Reworked internals to use non wrapping head and tail. Since they are
non wrapping, there is no issue with distinguishing between empty and
full. Since this appraoch would be vulnerable to wrapping on 32 bit
boundary, added a mechanism which periodically reduces all indexes to
avoid 32 bit wrapping.

After this rework, buffer has one byte more capacity. Simple test shows
slight performance improvement.

Updated tests to reflect increased capacity and added test to check if
it is possible to continuesly allocated 2 buffers of half ring buffer
size.

Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
2020-11-11 13:17:49 +01:00
Fabio Utzig
448db8087c doc: fix reference/util after sys/util.h split
Add back Doxygen documentation that previously existed in sys/util.h
and was moved to sys/util_macro.h.

Also fixes #29896

Signed-off-by: Fabio Utzig <fabio.utzig@nordicsemi.no>
2020-11-10 15:40:31 +01:00
Trond Einar Snekvik
ed1f75da74 sys: util: Add clamp macro
Adds CLAMP macro to complement the current min/max macros, as well as a
gcc specific Z_CLAMP macro for single-evaluation expansion.

CLAMP combines the functionality of MIN and MAX, eliminating the
bug-prone usage of MIN(MAX(value, FLOOR), CEIL) found throughout the
codebase in every possible combination.

Signed-off-by: Trond Einar Snekvik <Trond.Einar.Snekvik@nordicsemi.no>
2020-11-05 12:12:17 +01:00
Fabio Utzig
ab9602d401 doc: allow macro resolving in \deprecated usage
Breathe>=4.23.0 renders \xrefitem, including its specializations like
\deprecated, so add the proper `()` to symbols existing in previously
declared \deprecated messages so they are properly resolved.

Signed-off-by: Fabio Utzig <fabio.utzig@nordicsemi.no>
2020-10-27 13:09:24 +01:00
Daniel Leung
62cf1960ad kernel: support using thread local storage for errno
This enables storing errno in the thread local storage area.
With this enabled, a syscall to access errno can be avoided
when userspace is also enabled.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-10-24 10:52:00 -07:00
Daniel Leung
388725870f arm: cortex_m: add support for thread local storage
Adds the necessary bits to initialize TLS in the stack
area and sets up CPU registers during context switch.

Note that since Cortex-M does not have the thread ID or
process ID register needed to store TLS pointer at runtime
for toolchain to access thread data, a global variable is
used instead.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-10-24 10:52:00 -07:00
Daniel Leung
97db4ac0ca libc: add hidden option CONFIG_NEED_LIBC_MEM_PARTITION
The z_libc_partition was only enabled when newlib is being used,
and/or stack canaries are needed. This adds a hidden option
where this partition can be enabled if needed, regardless of
whether newlib is used or stack canaries are needed.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-10-24 10:52:00 -07:00
Gerson Fernando Budke
6b05121e65 include: sys: util: Move macros from util to util_macro
The <sys/util.h>, in current form, can not be used with DTS as it
contains non-C pre-processor definitions which breake DTS interpreter.
This commit fixes the problem by moving most of preprocessor macros
from util.h to util_macro.h.  Since util_mcaro.h contains only
preprocessor macros, without include dependencies, it can be safely
included in DTS. It is similar way the
<dts/arm/<manufacturer>/pinctrl_<manufacturer>_<soc>.h files are
included.

This fix and align the extern "C" closing brack inside non assembly
block.

The issue was raised when try create a macro for pincrtl with a
variable length flag list.

Signed-off-by: Gerson Fernando Budke <nandojve@gmail.com>
2020-10-23 09:38:55 -05:00
Andrew Boie
afe42bd129 arch_interface: update arch_mem_domain_thread_add
Implementations may assume the target thread isn't already
a member of the memory domain.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-10-22 16:47:07 -07:00
Andrew Boie
941322c49e userspace: document multiple mem domain init calls
k_mem_domain objects should not be initialized multiple times,
there's no support for memory domain life-cycles or freeing
memory that an arch_mem_domain_init() may have allocated.
Clearly document this.

The init function has to assume the provided domain is un-
initialized memory so it's not possible to robustly check
for this; a note is left in the arch_ definition to add an
assertion if feasible.

It's really unsafe to call an init function on any kernel
object more than once, but in this particular case if the
memory domain initialization resulted in the creation of
page tables or linked data structures really bad things can
happen. Currently no arch implements arch_mem_domain_init()
yet, but this is changing soon for x86.

One test case currently does this, it will be fixed in
a forthcoming patch.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-10-22 15:35:25 -04:00
Maximilian Bachmann
f219a982ff include: sys: use C linkage
add extern C when C++ is used to treat the code as C

Signed-off-by: Maximilian Bachmann <m.bachmann@acontis.com>
2020-10-22 07:49:07 -05:00
Andy Ross
a8d5437799 soc/xtensa: Misc. checkpatch fixups
Code style fixes.  Kept separate from the original changes to permit
easier rebasing.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2020-10-21 06:38:53 -04:00
Andy Ross
f6d32ab0a4 kernel: Add cache coherence management framework
Zephyr SMP kernels need to be able to run on architectures with
incoherent caches.  Naive implementation of synchronization on such
architectures requires extensive cache flushing (e.g. flush+invalidate
everything on every spin lock operation, flush on every unlock!) and
is a performance problem.

Instead, many of these systems will have access to separate "coherent"
(usually uncached) and "incoherent" regions of memory.  Where this is
available, place all writable data sections by default into the
coherent region.  An "__incoherent" attribute flag is defined for data
regions that are known to be CPU-local and which should use the cache.
By default, this is used for stack memory.

Stack memory will be incoherent by default, as by definition it is
local to its current thread.  This requires special cache management
on context switch, so an arch API has been added for that.

Also, when enabled, add assertions to strategic places to ensure that
shared kernel data is indeed coherent.  We check thread objects, the
_kernel struct, waitq's, timeouts and spinlocks.  In practice almost
all kernel synchronization is built on top of these structures, and
any shared data structs will contain at least one of them.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-10-21 06:38:53 -04:00
Peter Bigot
3dd804fe91 sys: util: add identity macro
Several utility macros must be given the name of a macro to apply to
every element in a sequence to transform it.  When no transformation
is necessary a macro still needs to be provided.  That macro is
IDENTITY.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2020-10-19 18:14:11 +02:00
Andrew Boie
95bbc742ba kernel: move kernel object APIs to own header
Part of an effort to break up the gigantic kernel.h.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-10-12 12:21:46 -04:00
Kumar Gala
014342a6f6 sys: ring_buffer: remove deprecated macros
Remove SYS_RING_BUF_DECLARE_POW2 and SYS_RING_BUF_DECLARE_SIZE as they
have been marked deprecated for at least 2 releases.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-10-06 14:58:42 -04:00
Aastha Grover
83b9f69755 code-guideline: Fixing code violation 10.4 Rule
Both operands of an operator in the arithmetic conversions
performed shall have the same essential type category.

Changes are related to converting the integer constants to the
unsigned integer constants

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2020-10-01 17:13:29 -04:00
Eric Johnson
c27561d984 sys: byteorder: Fix incorrect shift in sys_get_be48/le48
Fixes an incorrect left-shift value that was introducing
extra 0x00 bytes to the result.

Signed-off-by: Eric Johnson <eric@liveathos.com>
2020-09-21 11:17:59 +02:00
Fabio Utzig
39fa56b834 doc: update Kconfig options to use @option ALIAS
This updates Kconfig options in the Doxygen documentation to use the new
@option ALIAS. There are three categories of fixes:

* Use of `:option:` inside Doxygen headers, which is not valid (this is
  rST syntax!).
* Kconfig options that were just written as plain text and were no
  references were generated.
* Use of `@rst` blocks where the only reason for using them was to have
  Kconfig options resolved.

Signed-off-by: Fabio Utzig <fabio.utzig@nordicsemi.no>
2020-09-15 15:25:01 +02:00
Anas Nashif
6e27478c3d benchmarking: remove execution benchmarking code
This code had one purpose only, feed timing information into a test and
was not used by anything else. The custom trace points unfortunatly were
not accurate and this test was delivering informatin that conflicted
with other tests we have due to placement of such trace points in the
architecture and kernel code.

For such measurements we are planning to use the tracing functionality
in a special mode that would be used for metrics without polluting the
architecture and kernel code with additional tracing and timing code.

Furthermore, much of the assembly code used had issues.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-09-05 13:28:38 -05:00
Andrew Boie
5e0b55c30e kernel: demote k_mem_map to z_mem_map
Memory mapping, for now, will be a private kernel API
and is not intended to be application-facing at this time.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-09-03 14:24:38 -04:00
Andrew Boie
7d32e9f9a5 mmu: support only identity RAM mapping
We no longer plan to support a split address space with
the kernel in high memory and per-process address spaces.
Because of this, we can simplify some things. System RAM
is now always identity mapped at boot.

We no longer require any virtual-to-physical translation
for page tables, and can remove the dual-mapping logic
from the page table generation script since we won't need
to transition the instruction point off of physical
addresses.

CONFIG_KERNEL_VM_BASE and CONFIG_KERNEL_VM_LIMIT
have been removed. The kernel's address space always
starts at CONFIG_SRAM_BASE_ADDRESS, of a fixed size
specified by CONFIG_KERNEL_VM_SIZE.

Driver MMIOs and other uses of k_mem_map() are still
virtually mapped, and the later introduction of demand
paging will result in only a subset of system RAM being
a fixed identity mapping instead of all of it.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-09-03 14:24:38 -04:00
Flavio Ceolin
5408f3102d debug: x86: Add gdbstub for X86
It implements gdb remote protocol to talk with a host gdb during the
debug session. The implementation is divided in three layers:

1 - The top layer that is responsible for the gdb remote protocol.
2 - An architecture specific layer responsible to write/read registers,
    set breakpoints, handle exceptions, ...
3 - A transport layer to be used to communicate with the host

The communication with GDB in the host is synchronous and the systems
stops execution waiting for instructions and return its execution after
a "continue" or "step" command. The protocol has an exception that is
when the host sends a packet to cause an interruption, usually triggered
by a Ctrl-C. This implementation ignores this instruction though.

This initial work supports only X86 using uart as backend.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2020-09-02 20:54:57 -04:00
Tomasz Bursztyka
6df8b3995e irq: Change dynamic API to take a constant parameter
All ISRs are meant to take a const struct device pointer, but to
simplify the change let's just move the parameter to constant and that
should be fine.

Fixes #27399

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2020-09-02 13:48:13 +02:00
Tomasz Bursztyka
84942e4fbc irq: Change offload API to take a constant parameter
All ISRs are meant to take a const struct device pointer, but to
simplify the change let's just move the parameter to constant and that
should be fine.

Fixes #27399

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2020-09-02 13:48:13 +02:00