Following the idiom used for system calls, add script support to read
the initial application binary to identify which devices are defined,
and to use their offset in the device array as their unique handle
rather than the externally-defined ordinal from devicetree. The
device dependency arrays are updated to use these handles.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Move the busy status from a global atomic bit sequence to atomic flags
in the device PM state. While this temporarily adds 4 bytes to each
PM structure the whole device PM infrastructure will be refactored and
it's likely the extra memory can be recovered.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Separate the state indicator of whether the initialization function
has been invoked from the success or failure of the initialization.
This allows precise confirmation that the device is ready (i.e. it has
been initialized, and that initialization succeeded).
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
This avoids the need for distinct object that uses flash to store its
initializer. Instead the state is initialized when the kernel is
starting up, before anything can reference it. In future refactoring
the PM state could be accessed directly without storing an extra
pointer in the static device state.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Initialize all device objects in a batch before invoking any code that
might try to reference data in them. This eliminates a race condition
enabled by the ability to resolve a device structure at build time,
and reference it from one device's initialization routine before the
device itself has been initialized.
While the device is pulled from the sys_init records rather than
static devices, all in-tree init_entry records that are associated
with devices are produced via Z_DEVICE_DEFINE(), so there should be no
static devices that would be missed by instead iterating over the
device records.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Some recent changes exposed some common "arch_switch() anti-patterns"
in various architectures. The documentation technically described
this all correctly, but probably wasn't as clear as it should have
been. Rewrite, making clear exactly what needs to happen and how the
fields should be interpreted.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
It was possible with pathological timing (see below) for the scheduler
to pick a cycle of threads on each CPU and enter the context switch
path on all of them simultaneously.
Example:
* CPU0 is idle, CPU1 is running thread A
* CPU1 makes high priority thread B runnable
* CPU1 reaches a schedule point (or returns from an interrupt) and
decides to run thread B instead
* CPU0 simultaneously takes its IPI and returns, selecting thread A
Now both CPUs enter wait_for_switch() to spin, waiting for the context
switch code on the other thread to finish and mark the thread
runnable. So we have a deadlock, each CPU is spinning waiting for the
other!
Actually, in practice this seems not to happen on existing hardware
platforms, it's only exercisable in emulation. The reason is that the
hardware IPI time is much faster than the software paths required to
reach a schedule point or interrupt exit, so CPU1 always selects the
newly scheduled thread and no deadlock appears. I tried for a bit to
make this happen with a cycle of three threads, but it's complicated
to get right and I still couldn't get the timing to hit correctly. In
qemu, though, the IPI is implemented as a Unix signal sent to the
thread running the other CPU, which is far slower and opens the window
to see this happen.
The solution is simple enough: don't store the _current thread in the
run queue until we are on the tail end of the context switch path,
after wait_for_switch() and going to reach the end in guaranteed time.
Note that this requires changing a little logic to handle the yield
case: because we can no longer rely on _current's position in the run
queue to suppress it, we need to do the priority comparison directly
based on the existing "swap_ok" flag (which has always meant
"yielded", and maybe should be renamed).
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The QUEUED state flag was managed separately from the run queue
insertion/deletion, and the logic (while AFAICT perfectly correct) was
tangled in a few places trying to keep them in sync. Put the
management of both behind a queue_thread()/dequeue_thread() API for
clarity. The ALWAYS_INLINE usage seems to be working to get the
compiler to condense the resulting multiple assignments. No behavior
change.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The "null out the switch handle and put it back" code in the swap
implementation is a holdover from some defensive coding (not wanting
to break the case where we picked our current thread), but it hides a
subtle SMP race: when that field goes NULL, another CPU that may have
selected that thread (which is to say, our current thread) as its next
to run will be spinning on that to detect when the field goes
non-NULL. So it will get the signal to move on when we revert the
value, when clearly we are still running on the stack!
In practice this was found on x86 which poisons the switch context
such that it crashes instantly.
Instead, be firm about state and always set the switch handle of a
currently running thread to NULL immediately before it starts running:
right before entering arch_switch() and symmetrically on the interrupt
exit path.
Fixes#28105
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Some legacy spots in our IPC layer (legally) pass a NULL wait queue to
pend(). Allow this in the coherence assertion.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The poll code uses a dummy wait queue so the threads have something to
block on, but the previous coherence pass (which rearranged things to
put the _poller data elsewhere) missed that this was on the stack,
which is not allowed. It actually has no use except as a list, so
make it a global static instead.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The z_swap_unlocked() function used a dummy spinlock for simplicity.
But this runs afouls of checking for stack-resident spinlocks
(forbidden when KERNEL_COHERENCE is set). And it's executing needless
code to release the lock anyway. Replace with a compile time NULL,
which will improve performance, correctness and code size.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The two calls to unpend a thread from a wait queue were inexplicably*
unsynchronized, as James Harris discovered. Rework them to call the
lowest level primities so we can wrap the process inside the scheduler
lock.
Fixes#32136
* I took a brief look. What seems to have happened here is that these
were originally synchronized via an implicit from an outer caller
(remember the original Uniprocessor irq_lock() API is a recursive
lock), and they were mostly implemented in terms of middle-level
calls that were themselves locked. So those got ported over to the
newer spinlock but the outer wrapper layer got forgotten.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
This lets the linker tell us what kind of alignment is required
for both tdata and tbss data when copying them into stack.
If they are not aligned as expected by the toolchain, generated
code would be accessing incorrect location for thread variables.
Fixes#32015
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The linker script defines `z_mapped_size` as follows:
```
z_mapped_size = z_mapped_end - z_mapped_start;
```
This is done with the belief that precomputed values at link time will
make the code smaller and faster.
On Aarch64, symbol values are relocated and loaded relative to the PC
as those are normally meant to be memory addresses.
Now if you have e.g. `CONFIG_SRAM_BASE_ADDRESS=0x2000000000` then
`z_mapped_size` might still have a reasonable value, say 0x59334.
But, when interpreted as an address, that's very very far from the PC
whose value is in the neighborhood of 0x2000000000. That overflows the
4GB relocation range:
```
kernel/libkernel.a(mmu.c.obj): in function `z_mem_manage_init':
kernel/mmu.c:527:(.text.z_mem_manage_init+0x1c):
relocation truncated to fit: R_AARCH64_ADR_PREL_PG_HI21
```
The solution is to define `Z_KERNEL_VIRT_SIZE` in terms of
`z_mapped_end - z_mapped_start` at the source code level. Given this
is used within loops that already start with `z_mapped_start` anyway,
the compiler is smart enough to combine the two occurrences and
dispense with a size counter, making the code effectively
slightly better for all while avoiding the Aarch64 relocation
overflow:
```
text data bss dec hex filename
1216 8 294936 296160 484e0 mmu.c.obj.arm64.before
1212 8 294936 296156 484dc mmu.c.obj.arm64.after
1110 8 9244 10362 287a mmu.c.obj.x86-64.before
1106 8 9244 10358 2876 mmu.c.obj.x86-64.after
```
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The SYS_CLOCK_TICKS_PER_SEC default may depend on the kernel config
for tickless, rather than the capability.
Signed-off-by: Martin Åberg <martin.aberg@gaisler.com>
Activating K_FP_REGS flags introduces stack memory
overhead for the main thread in Cortex-M architecture.
Several ARM platforms experience main thread stack
overflows when building with FPU_SHARING=y.
Enabling FPU sharing in main thread should not be
the default configuration. Users are welcome to
enable FP sharing on the main thread in the
application code, in main().
This reverts commit 8453a73ede.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
The call to arch_mem_coherent() inside spinlock.h
when spinlock validation and memory coherence enabled
is causing build error as spinlock.h does not include
kernel_arch_func.h directly. However, simply including
that file does not work either as this creates
the chicken-or-egg in the chain of include files.
In order to make spin validation work with kernel
coherence enabled, a separate function is created
to break the circular dependencies of include files.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
There was an edge case in the timeout handling (exposed by, but not
strictly related to, the recent timeslice fix): the next_timeout()
computation would include time slice expiration as a clamp on the
result, but this would be invoked also on the z_set_timeout_expiry()
path which gets hooked on entry to a new thread which is needed to set
the timeout in the first place. So if no other timer interrupt was
scheduled, it was possible to miss the first timeslice interrupt after
thread scheduling.
The explanation is much longer than the fix (use <= as the comparator
instead of <).
In practice this was only being hit in the existing test suite on
riscv miv running under renode using non-default clock rates.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Fix an edge case that snuck in with the recent fix: if timeslicing is
enabled, the CPU's slice_ticks will be zero, and thus match a timeout
object's dticks value of zero, and thus get suppressed (because "we
already have a timeout scheduled for that") incorrectly.
Fixes#31789
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
There are more and more tests that fail due to stackoverflow.
Increasing MAIN_STACK_SIZE to fix those issues.
Signed-off-by: Alexandre Bourdiol <alexandre.bourdiol@st.com>
Time slices don't have a timeout struct associated and stored in
timeout_list. Time slice timeout is direct programmed in the system
clock and tracked in _current_cpu->slice_ticks.
There is one issue where the time slice timeout can be missed because
the system clock is re-programmed to a longer timeout. To this happens,
it is only necessary that the timeout_list is empty (any timeout set)
and a new timeout longer than remaining time slice is set. This is cause
because z_add_timeout does not check for the slice ticks.
The following example spots the issue:
K_THREAD_STACK_DEFINE(tstack, STACK_SIZE);
K_THREAD_STACK_ARRAY_DEFINE(tstacks, NUM_THREAD, STACK_SIZE);
K_SEM_DEFINE(sema, 0, NUM_THREAD);
static inline void spin_for_ms(int ms)
{
uint32_t t32 = k_uptime_get_32();
while (k_uptime_get_32() - t32 < ms) {
}
}
static void thread_time_slice(void *p1, void *p2, void *p3)
{
printk("thread[%d] - Before spin\n", (int)(uintptr_t)p1);
/* Spinning for longer than slice */
spin_for_ms(SLICE_SIZE + 20);
/* The following print should not happen before another
* same priority thread starts.
*/
printk("thread[%d] - After spinning\n", (int)(uintptr_t)p1);
k_sem_give(&sema);
}
void main(void)
{
k_tid_t tid[NUM_THREAD];
struct k_thread t[NUM_THREAD];
uint32_t slice_ticks = k_ms_to_ticks_ceil32(SLICE_SIZE);
int old_prio = k_thread_priority_get(k_current_get());
/* disable timeslice */
k_sched_time_slice_set(0, K_PRIO_PREEMPT(0));
for (int j = 0; j < 2; j++) {
k_sem_reset(&sema);
/* update priority for current thread */
k_thread_priority_set(k_current_get(), K_PRIO_PREEMPT(j));
/* synchronize to tick boundary */
k_usleep(1);
/* create delayed threads with equal preemptive priority */
for (int i = 0; i < NUM_THREAD; i++) {
tid[i] = k_thread_create(&t[i], tstacks[i], STACK_SIZE,
thread_time_slice, (void *)i, NULL,
NULL, K_PRIO_PREEMPT(j), 0,
K_NO_WAIT);
}
/* enable time slice (and reset the counter!) */
k_sched_time_slice_set(SLICE_SIZE, K_PRIO_PREEMPT(0));
/* Spins for while to spend this thread time but not longer */
/* than a slice. This is important */
spin_for_ms(100);
printk("before sleep\n");
/* relinquish CPU and wait for each thread to complete */
k_sleep(K_TICKS(slice_ticks * (NUM_THREAD + 1)));
for (int i = 0; i < NUM_THREAD; i++) {
k_sem_take(&sema, K_FOREVER);
}
/* test case teardown */
for (int i = 0; i < NUM_THREAD; i++) {
k_thread_abort(tid[i]);
}
/* disable time slice */
k_sched_time_slice_set(0, K_PRIO_PREEMPT(0));
}
k_thread_priority_set(k_current_get(), old_prio);
}
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Some arches like x86 need all memory mapped so that they can
fetch information placed arbitrarily by firmware, like ACPI
tables.
Ensure that if this is the case, the kernel won't accidentally
clobber it by thinking the relevant virtual memory is unused.
Otherwise this has no effect on page frame management.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
If we evict enough pages to completely fill the backing store,
through APIs like k_mem_map(), z_page_frame_evict(), or
z_mem_page_out(), this will produce a crash the next time we
try to handle a page fault.
The backing store now always reserves a free storage location
for actual page faults.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This will enable testing of the implementation until the
critical set of pages is identified and known to the
kernel.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Implement runtime APIs for pinning, paging in, and evicting
memory, as well as the page fault hook called from architecture
code.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Architecture layer hooks for demand paging. See
doxygen for these API definitions for more details.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Page tables created at build time may not include the
gperf data at the very end of RAM. Ensure this is mapped
properly at runtime to work around this.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Pre-allocation of paging structures is now required, such that
no allocations are ever needed when mapping memory.
Instantiation of new memory domains may still require allocations
unless a common page table is used.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Allows applications to increase the data space available to Zephyr
via anonymous memory mappings. Loosely based on mmap().
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The strategy used in z_heap_aligned_alloc() was to allocate an extra
align-sized memory block for storing a pointer to the memory heap.
This is wasteful in terms of memory usage when alignment is larger
than a pointer width. A loop is needed to find the initial memory
start when freeing it which isn't optimal either.
Instead, let's have sys_heap_aligned_alloc() rewind a pointer after
it is aligned to make just enough room for storing our heap reference.
This way the heap reference is always located immediately before the
aligned memory and any unused memory is returned to the heap.
The rewind and alignment values may coincide in which case only
the alignment is necessary anyway.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Remove conditionals (PM_DEEP_SLEEP_STATES and PM_SLEEP_STATES) from
power management code. Now these features are always available when
power management is enabled.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Migrate the whole pm subsystem to use new power states information
from power_state.h and get states and residency properties from
device tree.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
The internal API to measure time until a delay expires does not modify
the referenced timeout. Make the functions that call it take pointers
to const objects, so that they can be used with pointer to
const-qualified containers.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
This removes the z_ prefix those (functions, enums, etc.) that
are being used outside the coredump subsys. This aligns better
with the naming convention.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
If we evict enough pages to completely fill the backing store,
through APIs like k_mem_map(), z_page_frame_evict(), or
z_mem_page_out(), this will produce a crash the next time we
try to handle a page fault.
The backing store now always reserves a free storage location
for actual page faults.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This will enable testing of the implementation until the
critical set of pages is identified and known to the
kernel.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Implement runtime APIs for pinning, paging in, and evicting
memory, as well as the page fault hook called from architecture
code.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Architecture layer hooks for demand paging. See
doxygen for these API definitions for more details.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Page tables created at build time may not include the
gperf data at the very end of RAM. Ensure this is mapped
properly at runtime to work around this.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Pre-allocation of paging structures is now required, such that
no allocations are ever needed when mapping memory.
Instantiation of new memory domains may still require allocations
unless a common page table is used.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Allows applications to increase the data space available to Zephyr
via anonymous memory mappings. Loosely based on mmap().
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Following the idiom used for system calls, add script support to read
the initial application binary to identify which devices are defined,
and to use their offset in the device array as their unique handle
rather than the externally-defined ordinal from devicetree. The
device dependency arrays are updated to use these handles.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
The only two supported operations for data caches in the cache framework
are currently arch_dcache_flush() and arch_dcache_invd().
This is quite restrictive because for some architectures we also want to
control i-cache and in general we want a finer control over what can be
flushed, invalidated or cleaned. To address these needs this patch
expands the set of operations that can be performed on data and
instruction caches, adding hooks for the operations on the whole cache,
a specific level or a specific address range.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
register_event always returns 0, so the conditional will
always take the first branch and code in the else part
is never reached.
Fixes#31282
Signed-off-by: Ningx Zhao <ningx.zhao@intel.com>
1. Exclude the CODE UNREACHABLE line while generating coverage report.
2. Exclude the memory domain deprecated API when calculating code
coverage.
Signed-off-by: Enjia Mai <enjiax.mai@intel.com>
First, the maximum heap size must fit in 31 bits worth of chunks
because the internal 32-bit field holding the size is shared with
the `used` bit.
Then the mention of a 256-byte block in the doc is no longer
relevant. That pertained to the previous allocator implementation.
And ditto for the HEAP_MEM_POOL_MIN_SIZE kconfig option.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Needing to check the current cycle time (which involves a spinlock and
register read on most architectures) is wasteful in the scheduler
priority predicate, which is a hot path. If we "burn" one bit of
precision (and document the rule), we can do the comparison without
knowing the current time.
2^31 cycles is still far longer than a live deadline thread in any
legitimate realtime app should ever live before being scheduled.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Adds a linker section for Cortex-M instruction tightly coupled memory
(ITCM), similar to the existing section for DTCM. A new executable MPU
region is not added as there isn't currently a need to make this section
accessible to user mode. This section can be enabled by setting a device
tree chosen node zephyr,itcm.
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
This allows allocating dynamic kernel objects with memory alignment
requirements. The first candidate is for thread objects where,
on some architectures, it must be aligned for saving/restoring
registers.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
PM depends on SYS_CLOCK_EXISTS in Kconfig but several boards have
Kconfig overrides that allow the dependency to be ignored, so
CONFIG_PM=y even though CONFIG_SYS_CLOCK_EXISTS=n. Fix the code so
that the true dependency is reflected in the generated code.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
This change adds z_heap_aligned_alloc() and k_aligned_alloc()
and changes z_heap_malloc() and k_malloc() to be small wrappers around
the aligned variants.
Fixes#29519
Signed-off-by: Christopher Friedt <chrisfriedt@gmail.com>
Ticks should be assigned directly to timeout value in case of
CONFIG_LEGACY_TIMEOUT_API=y, just as they were before referenced patch.
Fixes: 7a815d5d99 ("kernel: sched: Use k_ticks_t in z_tick_sleep")
Signed-off-by: Marcin Niestroj <m.niestroj@grinn-global.com>
Renamed to make its semantics clearer; this function maps
*physical* memory addresses and is not equivalent to
posix mmap(), which might confuse people.
mem_map test case remains the same name as other memory
mapping scenarios will be added in the fullness of time.
Parameter names to z_phys_map adjusted slightly to be more
consistent with names used in other memory mapping functions.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Inside the idle loop, in some configuration, IRQ is unlocked and
then immediately locked again. There is a side effect:
1. IRQ is unlocked in middle of the loop.
2. Another thread (A) can now run so idle thread is un-scheduled.
3. Thread A runs to its end and going through the thread
self-abort path.
4. Idle thread is rescheduled again, and continues to run
the remaining loop when it eventuall calls k_cpu_idle().
The "pending abort" path is not being executed on thread A
at this point.
5. Now, thread A is suspended, and the CPU is in idle waiting
for interrupts (e.g. timeouts).
6. Thread B is waiting to join on thread A. Since thread A has
not been terminated yet so thread B is waiting until
the idle thread runs again and starts executing from
the beginning of while loop.
7. Depending on how many threads are running and how active
the platform is, idle thread may not run again for a while,
resulting in thread B appearing to be stuck.
To avoid this situation, the unlock/lock pair in middle of
the loop is removed so no rescheduling can be done mid-loop.
When there is no thread abort pending, it simply locks IRQ
and calls k_cpu_idle(). This is almost identical to the idle
loop before the thread abort code was introduced (except
the check for cpu->pending_abort).
Fixes#30573
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
In order to release irq_offload semaphore outside kernel/thread.c, we
make it visible by modifying it non-static under ztest. This would be
needed such as when call irq_offload() to enter interrupt context and
a fatal error happened, then you have to release it in your fatal
handler, or the irq_offload will still be locked and no longer be
using again.
Signed-off-by: Enjia Mai <enjiax.mai@intel.com>
Cleanup code for power management and remove some duplication and
isolate power management code from the kernel code.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
- Remove SYS_ prefix
- shorten POWER_MANAGEMENT to just PM
- DEVICE_POWER_MANAGEMENT -> PM_DEVICE
and use PM_ as the prefix for all PM related Kconfigs
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
k_heap did not have an aligned alloc function, even though
this is supported by the internal sys_heap.
Signed-off-by: Maximilian Bachmann <m.bachmann@acontis.com>
These implemented a k_mem_pool in terms of the now universal k_heap
utility. That's no longer necessary now that the k_mem_pool API has
been removed.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The mailbox and msgq utilities had API variants that could pass old
mem_pool blocks through the data structure. That API is being
deprected (and the features were obscure), so remove the internal
support.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The k_mem_pool allocator is no more, and the z_mem_pool compatibility
API is going away. The internal allocator should be a k_heap always.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
These were implemented in terms of the mem_pool/block API directly
(for complicated reasons, the pointers returned from this API may have
been allocated from allocators other than the single system heap).
Have them use a k_heap instead.
Requires a tweak to one test which had hard-coded an assumption about
the header size.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Mark all k_mem_pool APIs deprecated for future code. Remaining
internal usage now uses equivalent "z_mem_pool" symbols instead.
Fixes#24358
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Remove the MEM_POOL_HEAP_BACKEND kconfig, treating it as true always.
Now the legacy mem_pool cannot be enabled and all usage uses the
k_heap/sys_heap backend.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Changed algorithm of checking for pending thread in mem_slab.
Check for pending thread now is done only if there is
no memory left.
Signed-off-by: Kamil Lazowski <Kamil.Lazowski@nordicsemi.no>
z_tick_sleep was using int32_t what could cause a possible overflow
when converting from k_ticks_t.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
This patch activates FPU feature for main thread if FPU related
configs (FPU, FPU_SHARING) are enabled.
Signed-off-by: Katsuhiro Suzuki <katsuhiro@katsuster.net>
With MMU features enabled, we are using 248 out of 256
available bytes on 32-bit. This is extremely uncomfortable, relax
to a larger value like several other arches.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Adds a K_DELAYED_WORK_DEFINE, matching the K_WORK_DEFINE macro, with
accompanying Z_DELAYED_WORK_INITIALIZER macro.
Makes k_delayed_work_init a static inline function, like its K_WORK
counterpart.
Signed-off-by: Trond Einar Snekvik <Trond.Einar.Snekvik@nordicsemi.no>
Move banner and boot delay handling out of init.c
The code for banner was all over the place in init.c making it
unreadable.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Most of kernel files where declaring os module without providing
log level. Because of that default log level was used instead of
CONFIG_KERNEL_LOG_LEVEL.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
Add new function to mem_slab API that enables user
to get maximum number of slabs used so far.
Signed-off-by: Kamil Lazowski <Kamil.Lazowski@nordicsemi.no>
Use GEN_OFFSET_SYM macro to genarate absolute symbols for the
_callee_saved struct and use these new symbols in the assembly code.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
This uses the timing functions to gather execution cycles of
threads. This provides greater details if arch/SoC/board
uses timer with higher resolution.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Since the tracing of thread being switched in/out has the same
instrumentation points, we can roll the tracing function calls
into the one for thread stats gathering functions.
This avoids duplicating code to call another function.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds the bits to gather the first thread runtime statictic:
thread execution time. It provides a rough idea of how much time
a thread is spent in active execution. Currently it is not being
used, pending following commits where it combines with the trace
points on context switch as they instrument the same locations.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Documentation for kconfig SYS_CLOCK_TICKS_PER_SEC has some outdated
recommendations. Changing them to align with other documentation
under kernel timing.
Fixes: #25482
Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
This legacy struct still had a non-standard name. Clean it up to
conform to currrent naming guidelines.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Fix the issue where the kernel poll code would place the tracking
struct on the caller stack and share it with other threads, thus
creating a cache coherence issue on systems where KERNEL_COHERENCE is
enabled.
This works by eliminating the thread backpointer in struct _poller and
simply placing the (now just two-byte!) struct directly into the
thread struct.
Note that this doesn't attempt to fix the API paradigm that the
natural way to structure a call to k_poll() is to use an array of
k_poll_events on the CALLER's stack. So it's likely that most
"typical" k_poll code is still going to have problems with
KERNEL_COHERENCE. But at least now the kernel internals aren't
fundamentally broken.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The poll code was playing this weird trick where the thread pointer in
the "struct _poller" object for a triggered work item. It would not
be a thread to wake up, but instead a pointer to the (non-polling)
thread operated by the work queue being triggered. The code would
never touch this thread, just use it as a way to get a pointer to the
enclosing work queue struct.
Just store the work queue pointer in the first place. It's much
simpler, and makes future modifications to remove that thread pointer
possible.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The triggered work scheme uses a trick where it overloads the thread
pointer field of the struct poller (which normally stores the ID of
the thread that is blocked in k_poll()) to be able to find the work
queue to which it will submit.
Give it its own pointer field to break this false dependency.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Somewhat weirdly, k_poll() can do one of two things when the set of
events is signaled: it can wake up a sleeping thread, or it can submit
an unrelated work item to a work queue. The difference in behaviors
is currently captured by a callback, but as there are only two it's
cleaner to put this into a "mode" enumerant. That also shrinks the
size of the data such that the poller struct can be moved somewhere
other than the calling stack.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Some platforms may have multiple RAM regions which are
dis-continuous in the physical memory map. We really want
these to be in a continuous virtual region, and we need to
stop assuming that there is just one SRAM region that is
identity-mapped.
We no longer use CONFIG_SRAM_BASE_ADDRESS and CONFIG_SRAM_SIZE
as the bounds of kernel RAM, and no longer assume in the core
kernel that these are identity mapped at boot.
Two new Kconfigs, CONFIG_KERNEL_VM_BASE and
CONFIG_KERNEL_RAM_SIZE now indicate the bounds of this region
in virtual memory.
We are currently only memory-mapping physical device driver
MMIO regions so we do not need virtual-to-physical calculations
to re-map RAM yet. When the time comes an architecture interface
will be defined for this.
Platforms which just have one RAM region may continue to
identity-map it.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Replaces all existing variants of value clamping with the MIN and MAX
macros with the CLAMP macro.
Signed-off-by: Trond Einar Snekvik <Trond.Einar.Snekvik@nordicsemi.no>
k_object_alloc(K_OBJ_THREAD) returns a usable struct k_thread pointer.
This pointer is 4 byte aligned. On x86 and x86_64 struct _thread_arch
has a member which requires alignment. Since this is currently not
supported k_object_alloc(K_OBJ_THREAD) now returns an error instead of
a misaligned pointer.
Signed-off-by: Maximilian Bachmann <m.bachmann@acontis.com>
Toolchains other than Zephyr SDK may not support generating
code with thread local storage. So limit TLS to Zephyr SDK
for now, and only enable TLS on other toolchains as needed.
Fixes#29541
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
k_queue_append and k_queue_alloc_append are in a race
condition.
Scenario:
Using either of the above mentioned functions from a
preemptive context, can lead to the element being
appended to the queue to be lost. The element loss happens
when the k_queue_append is interrupted at a critical point,
when the list only contains one element and k_queue_get is
called, before the k_queue_append is allowed to complete.
Fix:
Move sys_sflist_peek_tail inside queue_insert(). Add additional
bool paramenter to queue_insert(), to indicate append/prepend
operation.
Fixes#29257
Signed-off-by: iva kik <megatheriumiva@gmail.com>
For threads that run in supervisor mode for some time before
synchronously dropping to user mode, re-initialize the TLS
area to prevent leakage of potentially sensitive information.
We did this already for CONFIG_THREAD_USERSPACE_LOCAL_DATA
but not the new CONFIG_THREAD_LOCAL_STORAGE.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This enables storing errno in the thread local storage area.
With this enabled, a syscall to access errno can be avoided
when userspace is also enabled.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Adds the necessary bits to initialize TLS in the stack
area and sets up CPU registers during context switch.
Note that since Cortex-M does not have the thread ID or
process ID register needed to store TLS pointer at runtime
for toolchain to access thread data, a global variable is
used instead.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds the common struct fields and functions to support
the implementation of thread local storage in individual
architecture. This uses the thread stack to store TLS data.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Add kconfigs to indicate whether an architecture has support
for thread local storage (TLS), and to enable TLS in kernel.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Arches which have custom swap to main can have _current be
NULL very, very early in the boot process. Check this to
avoid an infinite loop of fatal errors.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The core kernel does not use this yet, but it will be later used
as part of infrastructure for memory-mapping stacks, as detailed
in #28899.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
In the MMU code mapping_pos is miscalculated when for example
SRAM_BASE_ADDRESS==0x40000000 and KERNEL_VM_SIZE==0xc0000000 getting a
mapping_pos of 0x0.
The problem is that we must cast the two values to uintptr_t before
casting the result to avoid the rollover to 0.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
For compatibility layers like CMSIS where thread objects
are drawn from a pool, provide a context pointer to the
exited thread object so it may be freed.
This is somewhat obscure and has no supporting APIs or
overview documentation and should be considered a private
kernel feature. Applications should really be using
k_thread_join() instead.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
If we call k_mem_domain_add_thread() to a memory domain
the thread already belongs to, do nothing.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The synchronous arch hook wasn't being invoked when
k_mem_domain_init() was called with a partition list.
This could result with regions not being programmed in
the page tables as expected.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The type for the thread index returned by thread_index_get() must be
casted to int when comparing with (-1). Directly using uintptr_t is
breaking the ARMv8 implementation where where the check (index != -1) is
verified also when no thread index is returned.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The k_poll implementation places a struct _poller on the stack and
shares it with other threads, which is incompatible with the
KERNEL_COHERENCE model of cached stacks.
Make this a hard build failure instead of a kconfig dependency for
clarity. The failures if a user actually enables both are subtle and
difficult to debug.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Zephyr SMP kernels need to be able to run on architectures with
incoherent caches. Naive implementation of synchronization on such
architectures requires extensive cache flushing (e.g. flush+invalidate
everything on every spin lock operation, flush on every unlock!) and
is a performance problem.
Instead, many of these systems will have access to separate "coherent"
(usually uncached) and "incoherent" regions of memory. Where this is
available, place all writable data sections by default into the
coherent region. An "__incoherent" attribute flag is defined for data
regions that are known to be CPU-local and which should use the cache.
By default, this is used for stack memory.
Stack memory will be incoherent by default, as by definition it is
local to its current thread. This requires special cache management
on context switch, so an arch API has been added for that.
Also, when enabled, add assertions to strategic places to ensure that
shared kernel data is indeed coherent. We check thread objects, the
_kernel struct, waitq's, timeouts and spinlocks. In practice almost
all kernel synchronization is built on top of these structures, and
any shared data structs will contain at least one of them.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Strictly speaking, any access to a mem domain or its
containing partitions should be serialized on this lock.
Architecture code may need to grab this lock if it is
using this data during, for example, context switches,
especially if they support SMP as locking interrupts
is not enough.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This deprecated API won't be removed for one more release,
ensure it doesn't put the kernel into a bad state as it
currently sets all the member thread domain assignment to
NULL which is not what we want.
Have it reassign all member threads to the default domain.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
When threads exited we were leaving dangling references to
them in the domain's mem_domain_q.
z_thread_single_abort() now calls into the memory domain
code via z_mem_domain_exit_thread() to take it off.
The thread setup code now invokes z_mem_domain_init_thread(),
avoiding extra checks in k_mem_domain_add_thread(), we know
the object isn't currently a member of a doamin.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
When CONFIG_ARCH_HAS_CUSTOM_BUSY_WAIT is not defined, cycles_to_wait
is calculated using a division operation. This calculation could take a
significant amount of time (a few microseconds on some architectures,
depending on the system clock).
In the special case of zero usec_to_wait, the function should return
immediately rather than spend time on calculations.
For example, in spi driver (spi_context.h, _spi_context_cs_control()),
k_busy_wait() can be called with zero delay. This can increase spi
transaction time significantly.
Another improvement, is moving the start_cycles initialization
before cycles_to_wait calculation, so the time it takes to calculate
cycles_to_wait will be taken into account.
Signed-off-by: David Komel <a8961713@gmail.com>
Although the documentation states that only work items that have not
completed the delay can be cancelled, in fact pending items can also
be removed from a work queue. Document that behavior
Also document the specific return value that will be obtained based on
the state of the work item at the time cancellation was attempted
using the current implementation.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
The implementation checks the work queue pointer before taking a lock
and entering code that, depending on build options, may fault if that
pointer is null. Move the check under the lock to ensure condition is
preserved. Then the check in work_cancel is no longer necessary since
condition is verified under lock at all call sites.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
The z_impl_k_stack_push() has a spinlock that is set after
stack member access, which could cause race conditions and
used multiple return statements. This commit
- moves the lock before the CHECKIF
- implements goto for flow of lock, reschedule, and unlock
- uses ret for single return at the end
Signed-off-by: Jennifer Williams <jennifer.m.williams@intel.com>
API that takes k_timer structures but doesn't change data in them is
updated to const-qualify the underlying object, allowing information
to be retrieved from contexts where the containing object is
immutable.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
API that takes _timeout structures but doesn't change data in them is
updated to const-qualify the underlying object, allowing information
to be retrieved from contexts where the containing object is
immutable.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
Both operands of an operator in the arithmetic conversions
performed shall have the same essential type category.
Changes are related to converting the integer constants to the
unsigned integer constants
Signed-off-by: Aastha Grover <aastha.grover@intel.com>
k_thread_create() works as expected on both uninitialized memory,
or threads that have completely exited.
However, horrible and difficult to comprehend things can happen if a
thread object is already being used by the kernel and
k_thread_create() is called on it.
Historically this has been a problem with test cases trying to be
parsimonious with thread objects and not properly cleaning up
after themselves. Add an assertion for this which should catch
both the illegal creation of a thread already active, or threads
racing to create the same thread object.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Fixes races where threads on another CPU are joining the
exiting thread, since it could still be running when
the joiners wake up on a different CPU.
Fixes problems where the thread object is still being
used by the kernel when the fn_abort() function is called,
preventing the thread object from being recycled or
freed back to a slab pool.
Fixes a race where a thread is aborted from one CPU while
it self-aborts on another CPU, that was currently worked
around with a busy-wait.
Precedent for doing this comes from FreeRTOS, which also
performs final thread cleanup in the idle thread.
Some logic in z_thread_single_abort() rearranged such that
when we release sched_spinlock, the thread object pointer
is never dereferenced by the kernel again; join waiters
or fn_abort() logic may free it immediately.
An assertion added to z_thread_single_abort() to ensure
it never gets called with thread == _current outside of an ISR.
Some logic has been added to ensure z_thread_single_abort()
tasks don't run more than once.
Fixes: #26486
Related to: #23063#23062
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Deprecate the Kconfig option for the time being. Unless a contributor
volunteers to take over the work to maintain the option, it will be
removed after 2 releases.
Relates to #27415.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Amend Doxygen documentation for k_msgq_put to note that the message
content will not be modified as a result of the function call and
constify data parameter in function prototype.
Fixes#22301
Signed-off-by: Lauren Murphy <lauren.murphy@intel.com>
Adds a kconfig CONFIG_KERNEL_MEM_POOL to decide whether
kernel memory pool related code is compiled. This option
can be disabled to shrink code size. If k_heap is not
being used at all, kheap.c will also not be compiled,
resulting in further smaller code size.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When to->dticks is an int64_t it may happen that the calculated
remaining time is a value that cannot be exactly represented in the
destination int32_t, producing an implementation-defined result which
can include a signal (interrupt). Cap the maximum delay to the
largest value suported by the int32_t result.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
sys_trace_thread_abort and z_thread_monitor_exit in
z_thread_single_abort also need to be protected by
sched_spinlock, otherwise when after the spinlock
release, if there is an pending interrupt, it will cause an
sched in the interrrupt exit, and those trace and monitor
function will never reach.
Signed-off-by: Watson Zeng <zhiwei@synopsys.com>
We are checking thread->base members like thread_state and prio
and making decisions based on it, hold the sched_spinlock to
avoid potential concurrency problems if these members are modified
on another CPU or nested interrupt.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
We need to check if a thread is runnable at all before we
contemplate putting it on the end of the priority queue,
it might not be on the queue at all if it was suspended.
Replaces the less comprehensive check to see if the thread
was pending a timeout.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
We are checking and modifying members of thread->base
(in particular it's waitq and thread_state) which are
nominally protected by sched_spinlock. Hold it while
doing this to avoid concurrent changes on another CPU
or ISR preeemption.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The device status ready bit replaced the previous hack of clearing the
device API pointer when initialization of the device failed. It
inadvertently changed the behavior of device_get_binding() when
invoked before the device was initialized: previously that succeeded
for uninitialized devices, after the change it failed.
Multiple driver initializations rely on being able to get a device
pointer for something they're going to depend on in their init
function, even if that device has not yet been initialized. Although
this is wrong, and would cause faults if the device failed to
initialize before use, in practice it has been working.
It's not feasible to identify all the situations where this has
occurred, nor to add code to diagnose such cases without changing the
state associated with a device to distinguish initialized from
initialization success/failure. Restore the previous behavior until a
more holistic solution is developed.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
This code had one purpose only, feed timing information into a test and
was not used by anything else. The custom trace points unfortunatly were
not accurate and this test was delivering informatin that conflicted
with other tests we have due to placement of such trace points in the
architecture and kernel code.
For such measurements we are planning to use the tracing functionality
in a special mode that would be used for metrics without polluting the
architecture and kernel code with additional tracing and timing code.
Furthermore, much of the assembly code used had issues.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Add timing functions and APIs. This is now used with some of the tests
we have for performance and metrics and will be used whereever timing
informations are needed, for example for tracing, profiling and other
operations where timing info is critical.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
* Move switched_in into the arch context switch assembly code,
which will correctly record the switched_in information.
* Add switched_in/switched_out for context switch in irq exit.
Signed-off-by: Watson Zeng <zhiwei@synopsys.com>
Memory mapping, for now, will be a private kernel API
and is not intended to be application-facing at this time.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
We no longer plan to support a split address space with
the kernel in high memory and per-process address spaces.
Because of this, we can simplify some things. System RAM
is now always identity mapped at boot.
We no longer require any virtual-to-physical translation
for page tables, and can remove the dual-mapping logic
from the page table generation script since we won't need
to transition the instruction point off of physical
addresses.
CONFIG_KERNEL_VM_BASE and CONFIG_KERNEL_VM_LIMIT
have been removed. The kernel's address space always
starts at CONFIG_SRAM_BASE_ADDRESS, of a fixed size
specified by CONFIG_KERNEL_VM_SIZE.
Driver MMIOs and other uses of k_mem_map() are still
virtually mapped, and the later introduction of demand
paging will result in only a subset of system RAM being
a fixed identity mapping instead of all of it.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
It implements gdb remote protocol to talk with a host gdb during the
debug session. The implementation is divided in three layers:
1 - The top layer that is responsible for the gdb remote protocol.
2 - An architecture specific layer responsible to write/read registers,
set breakpoints, handle exceptions, ...
3 - A transport layer to be used to communicate with the host
The communication with GDB in the host is synchronous and the systems
stops execution waiting for instructions and return its execution after
a "continue" or "step" command. The protocol has an exception that is
when the host sends a packet to cause an interruption, usually triggered
by a Ctrl-C. This implementation ignores this instruction though.
This initial work supports only X86 using uart as backend.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
The same code was being copypasted in k_thread_abort()
implementations, just move into z_thread_single_abort().
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Now that device_api attribute is unmodified at runtime, as well as all
the other attributes, it is possible to switch all device driver
instance to be constant.
A coccinelle rule is used for this:
@r_const_dev_1
disable optional_qualifier
@
@@
-struct device *
+const struct device *
@r_const_dev_2
disable optional_qualifier
@
@@
-struct device * const
+const struct device *
Fixes#27399
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
The k_object API associates mutable state structures with known kernel
objects to support userspace. The kernel objects themselves are not
modified by the API, and in some cases (e.g. device structures) may be
const-qualified. Update the API so that pointers to these const
kernel objects can be passed without casting away the const qualifier.
Fixes#27399
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
In order to make all device instances constant, driver_api pointer is
not set to NULL anymore if initialization failed.
Instead, have a bitfield dedicated to it.
Fixes#27399
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
Following are the changes to variable names that are matching
with tag names (Rule 5.7 violations)
In kernel.h, event_type is matching with a tag name in
lib/os/onoff.c. Added a _ prefix to event_type and
also to the macro argument names.
In userspace.c, *dyn_obj is matching with the tag name
dyn_obj in the file itslef. Changed it to dyn
In device.h, device_mmio.h, init.h and init.c,
changed the *device to dev. Except for one change in
init.h
Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
Saves us a few bytes of program text on arches that don't need
these implemented, currently all uniprocessor MPU-based systems.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
We make a policy change here: all threads are members of a
memory domain, never NULL. We introduce a default memory domain
for threads that haven't been assigned to or inherited another one.
Primary motivation for this change is better MMU support, as
one common configuration will be to maintain page tables at
the memory domain level.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This adds a very primitive coredump mechanism under subsys/debug
where during fatal error, register and memory content can be
dumped to coredump backend. One such backend utilizing log
module for output is included. Once the coredump log is converted
to a binary file, it can be used with the ELF output file as
inputs to an overly simplified implementation of a GDB server.
This GDB server can be attached via the target remote command of
GDB and will be serving register and memory content. This allows
using GDB to examine stack and memory where the fatal error
occurred.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Move tracing switched_in and switched_out to the architecture code and
remove duplications. This changes swap tracing for x86, xtensa.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The memory domain APIs document that overlapping regions are
not allowed, check for this unconditionally.
Cleanup assertion error messages. Use __ASSERT_NO_MSG for
blindingly obvious NULL checks.
We now have a `check_add_partition()` function to perform
all the necessary sanity checks on adding a partition to a
domain. This returns true or false, which will help later
when we implement #24609
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Some systems may need to associate arch-specific data to
a memory domain. Add a Kconfig and `arch` field for this,
and a new arch API to initialize it.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Zephyr has a policy of invoking the scheduler on the way
out of IRQs anyway for all arches, this is unnecessary.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The dticks field was changed from a signed to an unsigned
value so now the assert test to ensure it isn't negative
is no longer needed.
fixes#26355
Signed-off-by: David Leach <david.leach@nxp.com>
Check if next ready thread is same as current thread before calling
z_swap.
This avoids calling swap to just go back to the original thread.
Original code: thread 0x20000118 switches out and then in again...
>> 0x20000118 gives semaphore(signal): 0x20000104 (count: 0)
>> thread ready: 0x20000118
>> 0x20000118 switched out
>> 0x20000118 switched in
>> end call to k_sem_give
>> 0x20000118 takes semaphore(wait): 0x200000f4 (count: 0)
>> thread pend: 0x20000118
>> 0x20000118 switched out
>> 0x200001d0 switched in
with this patch:
>> 0x200001d0 gives semaphore(signal): 0x200000f4 (count: 0)
>> thread ready: 0x200001d0
>> end call to k_sem_give
>> 0x200001d0 takes semaphore(wait): 0x20000104 (count: 0)
>> thread pend: 0x200001d0
>> 0x200001d0 switched out
>> 0x20000118 switched in
>> end call to k_sem_take
The above is output from tracing with a custom format used for
debugging.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Certain architectures, such as ARM Cortex-M require a custom
switch to main(), in case the case we build Zephyr without
support for multithreading (CONFIG_MULTITHREADING=n). So as
to keep the change to kernel/init.c as unintrusive as
possible, we install an ARCH-specific hook in z_cstart(),
which gets called for architectures that require this
customized switch to main() function.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Include directories for ${ARCH} is not specified correctly.
Several places in Zephyr, the include directories are specified as:
${ZEPHYR_BASE}/arch/${ARCH}/include
the correct line is:
${ARCH_DIR}/${ARCH}/include
to correctly support out of tree archs.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
include/cache.h: System calls declaration and implementation
kernel/cache_handlers.c: Defination of verification functions
Signed-off-by: Aastha Grover <aastha.grover@intel.com>
Add k_delayed_work_pending similar to k_work_pending to check if the
delayed work item has been submitted but not yet completed.
This would compliment the API since using k_work_pending or
k_delayed_work_remaining_get is not enough to check this condition.
This is because the timeout could have run out, but the timeout handler
not yet processed and put the work into the workqueue.
Signed-off-by: Joakim Andersson <joakim.andersson@nordicsemi.no>
This set of functions seem to be there just because of historical
reasons, stemming from Kbuild. They are non-obvious and prone to errors,
so remove them in favor of the `_ifdef()` ones with an explicit
`CONFIG_` condition.
Script used:
git grep -l _if_kconfig | xargs sed -E -i
"s/_if_kconfig\(\s*(\w*)/_ifdef(CONFIG_\U\1\E \1/g"
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
These stacks are appropriate for threads that run purely in
supervisor mode, and also as stacks for interrupt and exception
handling.
Two new arch defines are introduced:
- ARCH_KERNEL_STACK_GUARD_SIZE
- ARCH_KERNEL_STACK_OBJ_ALIGN
New public declaration macros:
- K_KERNEL_STACK_RESERVED
- K_KERNEL_STACK_EXTERN
- K_KERNEL_STACK_DEFINE
- K_KERNEL_STACK_ARRAY_DEFINE
- K_KERNEL_STACK_MEMBER
- K_KERNEL_STACK_SIZEOF
If user mode is not enabled, K_KERNEL_STACK_* and K_THREAD_STACK_*
are equivalent.
Separately generated privilege elevation stacks are now declared
like kernel stacks, removing the need for K_PRIVILEGE_STACK_ALIGN.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This now takes a stack pointer as an argument with TLS
and random offsets accounted for properly.
Based on #24467 authored by Flavio Ceolin.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The core kernel computes the initial stack pointer
for a thread, properly aligning it and subtracting out
any random offsets or thread-local storage areas.
arch_new_thread() no longer needs to make any calculations,
an initial stack frame may be placed at the bounds of
the new 'stack_ptr' parameter passed in. This parameter
replaces 'stack_size'.
thread->stack_info is now set before arch_new_thread()
is invoked, z_new_thread_init() has been removed.
The values populated may need to be adjusted on arches
which carve-out MPU guard space from the actual stack
buffer.
thread->stack_info now has a new member 'delta' which
indicates any offset applied for TLS or random offset.
It's used so the calculations don't need to be repeated
if the thread later drops to user mode.
CONFIG_INIT_STACKS logic is now performed inside
z_setup_new_thread(), before arch_new_thread() is called.
thread->stack_info is now defined as the canonical
user-accessible area within the stack object, including
random offsets and TLS. It will never include any
carved-out memory for MPU guards and must be updated at
runtime if guards are removed.
Available stack space is now optimized. Some arches may
need to significantly round up the buffer size to account
for page-level granularity or MPU power-of-two requirements.
This space is now accounted for and used by virtue of
the Z_THREAD_STACK_SIZE_ADJUST() call in z_setup_new_thread.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
arch_new_thread() passes along the thread priority and option
flags, but these are already initialized in thread->base and
can be accessed there if needed.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
MISRA-C Rule 5.3 states that identifiers in inner scope should
not hide identifiers in outer scope.
In the function smp_init_top(), the variable "start_flags"
collide with global variable of the same name. So rename the one
inside the function.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
MISRA-C Rule 5.3 states that identifiers in inner scope should
not hide identifiers in outer scope.
In the function z_set_timeout_expiry(), the parameter "idle"
and an inner variable "next" collide with function named idle()
and next(). So rename those variables.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
MISRA-C Rule 5.3 states that identifiers in inner scope should
not hide identifiers in outer scope. There are a few spinlock
keys simply named "key". So rename those in inner scope to avoid
shadowing the outer ones.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The mempool implementation doesn't require specific sizes and can
support arbitrary sizes up to the limit of available memory. The
Kconfig documentation on this configuration was confusing user.
Fixes#20418
Signed-off-by: David Leach <david.leach@nxp.com>
Later this year I hope to overhaul the memory domain APIs,
but at least for now let's at least consolidate these checks.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This will be the interface for mapping memory in the kernel's
part of the address space, which is guaranteed to be persistent
regardless of what thread is scheduled.
Further code for specifically managing virtual memory will end up in
kernel/mmu.c.
Further defintions for memory management in general will end up
in sys/mem_manage.h.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Unit tests were failing to build because random header was included by
kernel_includes.h. The problem is that rand32.h includes a generated
file that is either not generated or not included when building unit
tests. Also, it is better to limit the scope of this file to where it is
used.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Add one another test case for testing both arch_curr_cpu() and
arch_sched_ipi() architecture layer interface.
Signed-off-by: Enjia Mai <enjiax.mai@intel.com>
Currently this is useful only for some internal applications that
iterate over the device table, since applications can't get access to
a device that isn't ready, and devices can't be made unready. So it's
introduced as internal API that may be exposed as device_ready() when
those conditions change.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Device objects in Zephyr are currently placed into an array by linker
scripts, making it easy to iterate over all devices if the array
address and size can be obtained. This has applications in device
power management, but the existing API for this was available only
when that feature was enabled. It also uses a signed type to hold the
device count.
Provide a new API that is generally available, but marked as internal
since normally applications should not iterate over all devices. Mark
the PM API approach deprecated.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
We now have a low-level function z_dynamic_object_create()
which is not a system call and is used for installing
kernel objects that are not supported by k_object_alloc().
Checking for valid object type enumeration values moved
completely to the implementation function.
A few debug messages and comments were improved.
Futexes and sys_mutexes are now properly excluded from
dynamic generation.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Certain types of system call validation may need to be pushed
deeper in the implementation and not performed in the verification
function. If such checks are only pertinent when the caller was
from user mode, we need an API to detect this situation.
This is implemented by having thread->syscall_frame be non-NULL
only while a user system call is in progress. The template for the
system call marshalling functions is changed to clear this value
on exit.
A test is added to prove that this works.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This was passing along _current->ssf, but these types of bad
syscalls do not go through the z_mrsh mechanism and was
passing stale data.
We have the syscall stack frame already as an argument,
propagate that so it works properly.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The k_queue data structure, when CONFIG_POLL was enabled, would
inexplicably use k_poll() as its blocking mechanism instead of the
original wait_q/pend() code. This was actually racy, see commit
b173e4353f. The code was structured as a condition variable: using
a spinlock around the queue data before deciding to block. But unlike
pend_current_thread(), k_poll() cannot atomically release a lock.
A workaround had been in place for this, and then accidentally
reverted (both by me!) because the code looked "wrong".
This is just fragile, there's no reason to have two implementations of
k_queue_get(). Remove.
Note that this also removes a test case in the work_queue test where
(when CONFIG_POLL was enabled, but not otherwise) it was checking for
the ability to immediately cancel a delayed work item that was
submitted with a timeout of K_NO_WAIT (i.e. "queue it immediately").
This DOES NOT work with the origina/non-poll queue backend, and has
never been a documented behavior of k_delayed_work_submit_to_queue()
under any circumstances. I don't know why we were testing this.
Fixes#25904
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The possibility of passing K_FOREVER as the initial duration argument
to k_timer_start() wasn't being handled, with the result that the
computed value became an zero timeout (effecitvely treating it as
K_NO_WAIT and firing at the next tick).
This is obviously pathlogical, but it should still do what the code
says it should and wait forever.
Make k_timer_start(..., K_FOREVER, ...) a noop.
Fixes#25820
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Mutex operations check ownership against _current. But in an
ISR, _current is just whatever thread was interrupted when the
ISR fired. Explicitly do not allow this.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The device_pm_control_nop function is documented to always return zero
regardless of operation. However, when device_get_power_state() is
invoked with this control function it returns success leaving the
output parameter state unmodified, which may not be a valid device
state.
Document and implement that the nop control function returns -ENOTSUP
always.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Ensures that TLS from when the thread was in supervisor mode
is erased, rather than rely on the arch code to do it.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
- simplify dummy thread initialization to a kswap.h
inline function
- use the same inline function for both early boot and
SMP setup
- add a note on necessity of the dummy thread even if
a custom swap to main is implemented
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Several reviewers agreed that DT_HAS_NODE_STATUS_OKAY(...) was an
undesirable API for the following reasons:
- it's inconsistent with the rest of the DT_NODE_HAS_FOO names
- DT_NODE_HAS_FOO_BAR_BAZ(node) was agreed upon as a shorthand
for macros which are equivalent to
DT_NODE_HAS_FOO(node) && DT_NODE_HAS_BAR(node) &&
- DT_NODE_HAS_BAZ(node), and DT_HAS_NODE_STATUS_OKAY is an odd duck
- DT_NODE_HAS_STATUS(..., okay) was viewed as more readable anyway
- it is seen as a somewhat aesthetically challenged name
Replace all users with DT_NODE_HAS_STATUS(..., okay), which is
semantically equivalent.
This is mostly done with sed, but a few remaining cases were done by
hand, along with whitespace, docs, and comment changes. These special
cases include the Nordic SOC static assert files.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
C++ is documented to be supported in applications, so it should be
supported in SYS_INIT() functions run at the application init level.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
When the device driver model got introduced, there were no concept of
SYS_INIT() which can be seen as software service. These were introduced
afterwards and reusing the device infrastructure for simplicity.
However, it meant to allocate a bit too much for something that only
required an initialization function to be called at right time.
Thus refactoring the devices structures relevantly:
- introducing struct init_entry which is a generic init end-point
- struct deviceconfig is removed and struct device owns everything now.
- SYS_INIT() generates only a struct init_entry via calling
INIT_ENTRY_DEFINE()
- DEVICE_AND_API_INIT() generates a struct device and calls
INIT_ENTRY_DEFINE()
- init objects sections are in ROM
- device objects sections are in RAM (but will end up in ROM once they
will be 'constified')
It also generate a tiny memory gain on both ROM and RAM, which is nice.
Perhaps kernel/device.c could be renamed to something more relevant.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
This commit renames the Kconfig `FP_SHARING` symbol to `FPU_SHARING`,
since this symbol specifically refers to the hardware FPU sharing
support by means of FPU context preservation, and the "FP" prefix is
not fully descriptive of that; leaving room for ambiguity.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
These helper functions may clarify how much data is available to read
from or write to the circular buffer of a k_pipe.
Fixes#25036
Signed-off-by: Christopher Friedt <chrisfriedt@gmail.com>
Rename DT_HAS_NODE to DT_HAS_NODE_STATUS_OKAY so the semantics are
clear. As going forward DT_HAS_NODE will report if a NODE exists
regardless of its status.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
This was missing a prompt string, causing recent kconfig logic to
throw an error if an app tried to set it directly.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
