Commit Graph

300 Commits

Author SHA1 Message Date
Anas Nashif
4e396174ce kernel: move syscall_handler.h to internal include directory
Move the syscall_handler.h header, used internally only to a dedicated
internal folder that should not be used outside of Zephyr.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-11-03 11:46:52 +01:00
Anas Nashif
a6b490073e kernel: object: rename z_object -> k_object
Do not use z_ for internal structures and rename to k_object instead.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-11-03 11:46:52 +01:00
Anas Nashif
c91cad735a kernel: object: rename z_object_init to k_object_init
Do not use z_ for internal API and rename to k_object_init.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-11-03 11:46:52 +01:00
Flavio Ceolin
6e02f02464 kernel: Fix k_thread_name_get signature
k_thread_name_get has inconsistent signature. In the function
declaration it uses k_tid_t but in the implementation it is using
struct k_thread *. Change the implementation to use k_tid_t.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2023-10-30 17:34:45 -04:00
Flavio Ceolin
f9c7a5e6fb kernel: random: Rename early random get function
Rename z_early_boot_rand_get with z_early_rand_get to get consistent
with other early functions.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2023-10-13 10:03:53 +03:00
Keith Packard
362d8906a6 kernel: Support platforms with runtime timer freq for thread init_delay
Platforms that determine their basic timer frequency at runtime instead of
build time cannot compute thread initialization timeouts during
compilation.

Switch back to storing the init_delay value in milliseconds and perform the
conversion to a k_timeout_t at runtime.

Signed-off-by: Keith Packard <keithp@keithp.com>
2023-10-12 20:39:20 +03:00
Flavio Ceolin
e7bd10ae71 random: Rename random header
rand32.h does not make much sense, since the random subsystem
provides more APIs than just getting a random 32 bits value.

Rename it to random.h and get consistently with other
subsystems.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2023-10-10 14:23:50 +03:00
Peter Mitsis
e6f1090553 kernel: Integrate object core statistics
Integrates object core statistics framework into the following
kernel objects:
  sys_mem_blocks, k_mem_slab
  threads, _cpu, z_kernel

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2023-09-30 08:04:14 +03:00
Peter Mitsis
6df8efe354 kernel: Integrate object cores into kernel
Integrates object cores into the following kernel structures
   sys_mem_blocks, k_mem_slab
   _cpu, z_kernel
   k_thread, k_timer
   k_condvar, k_event, k_mutex, k_sem
   k_mbox, k_msgq, k_pipe, k_fifo, k_lifo, k_stack

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2023-09-30 08:04:14 +03:00
Keith Packard
41e8b44619 kernel: Make thread 'init_delay' k_timeout_t rather than int msecs
Storing this value in milliseconds rather than using k_timeout_t requires
the system to perform division at runtime to convert types. This pulls in
the 64-bit soft division code on platforms without hardware for this.

Perform the conversion at build time instead by using the runtime time
directly.

The init_delay field was moved within the _static_thread_data structure to
avoid introducing a hole for alignment on 32-bit systems when using 64-bit
timeouts.

Use SYS_TIMEOUT_MS instead of K_MSEC so that the initial delay can be set
to forever.

Signed-off-by: Keith Packard <keithp@keithp.com>
2023-09-28 16:15:27 +02:00
Benjamin Cabé
a46f1b9c33 kernel: Fix unused-parameter warnings
Add missing ARG_UNUSED where needed.

Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
2023-09-28 16:14:39 +02:00
Anas Nashif
8634c3b444 kernel: move wait_q.h header to be internal
This header does not expose any public APIs, so move it under
kernel/include and change files including it.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-09-12 12:55:36 -04:00
Gerard Marull-Paretas
dacb3dbfeb iterable_sections: move to specific header
Until now iterable sections APIs have been part of the toolchain
(common) headers. They are not strictly related to a toolchain, they
just rely on linker providing support for sections. Most files relied on
indirect includes to access the API, now, it is included as needed.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-05-22 10:42:30 +02:00
Aastha Grover
877fc3d508 kernel: events: fix waitq timeout race condition
Updates events to prevent a timeout from corrupting the list of
threads that needs to be waken up.

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2023-03-09 09:22:21 +01:00
Kumar Gala
a1195ae39b smp: Move for loops to use arch_num_cpus instead of CONFIG_MP_NUM_CPUS
Change for loops of the form:

for (i = 0; i < CONFIG_MP_NUM_CPUS; i++)
   ...

to

unsigned int num_cpus = arch_num_cpus();
for (i = 0; i < num_cpus; i++)
   ...

We do the call outside of the for loop so that it only happens once,
rather than on every iteration.

Signed-off-by: Kumar Gala <kumar.gala@intel.com>
2022-10-21 13:14:58 +02:00
Keith Packard
95cec04c7c kernel: When TLS, skip switched_out tracing hook from dummy thread
The dummy thread doesn't include a TLS area, so any thread local variables
will fail to work if used in the switched_out tracing hook. Skip the hook
in this case, as it's not really accurate anyways; the dummy thread is
only used to set context for the initial switch for each core.

Signed-off-by: Keith Packard <keithp@keithp.com>
2022-10-19 16:00:00 +02:00
Krzysztof Chruscinski
bf39f545e3 kernel: thread: Add casting to pointers in the log message
Using char pointers for %p should be avoided in log messages. It will
cause issues in configurations where logging strings are removed from
the binary and they are not inspected when cbprintf packages from
logging string are built. In that case any char pointers are treated as
strings and copied into the pacakge body.

Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
2022-10-18 14:13:19 +02:00
Peter Mitsis
976e4087ec kernel: Fix gathering of runtime thread stats
The function k_thread_runtime_stats_all_get() now populates the
current_cycles field in the thread runtime stats structure.

Resets the number of cycles in the CPU's current usage window once
the idle thread is scheduled.

Fixes the average_cycles calcuation.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2022-05-13 10:19:53 -05:00
Gerard Marull-Paretas
cffefc818d kernel: migrate includes to <zephyr/...>
In order to bring consistency in-tree, migrate all kernel code to the
new prefix <zephyr/...>. Note that the conversion has been scripted,
refer to zephyrproject-rtos#45388 for more details.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-05-09 09:26:20 +02:00
Peter Mitsis
a30cf39975 kernel: update k_thread_state_str() API
When threads are in more than one state at a time, k_thread_state_str()
returns a string that lists each of its states delimited by a '+'.
This in turn necessitates a change to the API that includes both a
pointer to the buffer to use for the string and the size of the buffer.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2022-04-20 20:20:13 -04:00
Dominik Ermel
8db0e5a42e kernel: Reduce strncpy in z_impl_k_thread_name_set by null char
No point to copy terminating NULL character to just overwrite it in
next line.

Signed-off-by: Dominik Ermel <dominik.ermel@nordicsemi.no>
2022-03-10 10:12:48 +01:00
Andy Ross
3e696896bf kernel: Add "per thread" timeslice mechanism
Zephyr's timeslice implementation has always been somewhat primitive.
You get a global timeslice that applies broadly to the whole bottom of
the priority space, with no ability (beyond that one priority
threshold) to tune it to work on certain threads, etc...

This adds an (optionally configurable) API that allows timeslicing to
be controlled on a per-thread basis: any thread at any priority can be
set to timeslice, for a configurable per-thread slice time, and at the
end of its slice a callback can be provided that can take action.
This allows the application to implement things like responsiveness
heuristics, "fair" scheduling algorithms, etc... without requiring
that facility in the core kernel.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2022-03-09 13:49:44 -05:00
Andy Ross
73453a39d1 arch: Add IRQ_OFFSET_NESTED feature
The x86 and xtensa implementations of irq_offload() invoke synchronous
interrupts on the local CPU, and are therefore safe to use from within
an interrupt context.  This is a cheap and portable way to exercise
nested interrupts, which are otherwise highly platform-dependent to
test.  Add a kconfig to signal the capability.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2022-02-21 22:10:03 -05:00
Krzysztof Chruscinski
1da97e1374 kernel: Add function for calculating stack usage
Extracting stack usage calculation from k_thread_stack_space_get to
z_stack_space_get so it can be used also for interrupt stack.

Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
2022-02-21 20:57:17 -05:00
Peter Mitsis
0ebd6c7f26 kernel/sched: enable/disable runtime stats
Adds support to enable/disable both thread and cpu runtime
stats.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2022-01-10 10:38:06 -05:00
Peter Mitsis
4eb1dd02cc kernel: extend CPU runtime stats
Extends the CPU usage runtime stats to track current, total, peak
and average usage (as bounded by the scheduling of the idle thread).
This permits a developer to obtain more system information if desired
to tune the system.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2022-01-10 10:38:06 -05:00
Peter Mitsis
5deaffb2ee kernel: update z_sched_thread_usage()
This commit does two things to the z_sched_thread_usage(). First,
it updates the API so that it accepts a pointer to the runtime
stats instead of simply returning the usage cycles. This gives it
the flexibility to retrieve additional statistics in the future.

Second, the runtime stats are only updated if the specified thread
is the current thread running on the current core.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2022-01-10 10:38:06 -05:00
Andy Ross
410f911018 kernel/sched: Separate idle from app thread stats in THREAD_USAGE
It turns out that we have a sample (though not a test) that really
does want to use "k_thread_runtime_stats_all_get()" to measure system
uptime.

Instead of breaking this needlessly, separate the accounting for idle
and non-idle threads.  The legacy API can report their sum, and the
more useful value is available via the kernel struct for future
analysis.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-11-08 21:32:20 -05:00
Andy Ross
f169c5bc13 kernel: Swap RUNTIME_STATS implementation
Clean up RUNTIME_STATS to separate the API from the individual data
backends.  Use the SCHED_THREAD_USAGE tracking instead of the original
for execution_cycles.  Move the kconfig for that into the runtime
stats menu, since it's part of the family now.

Also remove a lot of needless #if's around the declarations.  Unused
structs and uncalled functions don't need to be explicitly hidden.  An
attempt to access a non-existent field (e.g. "execution_cycles" if
that isn't configured) provides all the build time validation we need.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-11-08 21:32:20 -05:00
Andy Ross
4ae3250301 sched: Hook SCHED_USAGE from existing tracing hook
On older architectures, we don't have the
architecture-independent/scheduler-internal hooks (which require
USE_SWITCH) but there is a hook shared by the tracing layer we can use.

This is sort of a layering violation (stat tracking is a core feature,
tracing is supposed to be optional), but simple and lightweight.  And
eventually it will go away as these architectures migrate.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-11-08 21:32:20 -05:00
Chris Reed
f8f3c00d07 kernel: thread.c: remove unused THREAD_COOKIE macro.
This macro was not reference anywhere.

Signed-off-by: Chris Reed <chris.reed@arm.com>
2021-10-22 18:08:56 -04:00
Andy Ross
b11e796c36 kernel/sched: Add CONFIG_CPU_MASK_PIN_ONLY
Some SMP applications have threading designs where every thread
created is always assigned to a specific CPU, and never want to
schedule them symmetrically across CPUs under any circumstance.

In this situation, it's possible to optimize the run queue design a
bit to put a separate queue in each CPU struct instead of having a
single global one.  This is probably good for a few cycles per
scheduling event (maybe a bit more on architectures where cache
locality can be exploited) in circumstances where there is more than
one runnable thread.  It's a mild optimization, but a basically simple
one.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-09-28 20:15:05 -04:00
Krzysztof Chruscinski
76c309db7b kernel: thread: Fix thread runtime stats
Adding missing parenthesis. Without them wrong results
appeared when k_cycle_get_32 wrapped.

Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
2021-09-23 11:31:02 -04:00
Fabio Baltieri
f88a420d69 toolchain: migrate iterable sections calls to the external API
This migrates all the current iterable section usages to the external
API, dropping the "Z_" prefix:

Z_ITERABLE_SECTION_ROM
Z_ITERABLE_SECTION_ROM_GC_ALLOWED
Z_ITERABLE_SECTION_RAM
Z_ITERABLE_SECTION_RAM_GC_ALLOWED
Z_STRUCT_SECTION_ITERABLE
Z_STRUCT_SECTION_ITERABLE_ALTERNATE
Z_STRUCT_SECTION_FOREACH

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2021-08-12 17:47:04 -04:00
Maksim Masalski
e9600a75fc kernel: thread: add default case, remove unused break
According to the Zephyr Coding Guideline all switch statements
shall be well-formed. Add a default case with break and comment
to avoid static analysis tool to raise a violation that there is no
default case.
Also, I think, in all cases above no need to use "break",
because they already are using "return".

Found as a coding guideline violation (MISRA R16.1) by static
coding scanning tool.

Signed-off-by: Maksim Masalski <maksim.masalski@intel.com>
2021-06-04 16:21:01 -05:00
Ioannis Glaropoulos
dd574f6ec7 kernel: stack_sentinel: disable in single-threaded builds
Add a dependency on MULTITHREADING for the
STACK_SENTINEL feature, so it may not get
enabled in single-thread Zephyr builds.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2021-05-28 10:41:46 -05:00
Torbjörn Leksell
f17144349b Tracing: Thread tracing
Add thread tracing hooks, default hooks, and documentation.

Signed-off-by: Torbjörn Leksell <torbjorn.leksell@percepio.com>
2021-05-07 22:10:21 -04:00
Krzysztof Chruscinski
b8fb353cd4 kernel: Move k_busy_wait from thread to timeout
K_busy_wait is the only function from thread.c that is used when
CONFIG_MULTITHREADING=n. Moving to timeout since it fits better there
as it requires sys clock to be present.

Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
2021-04-29 14:50:35 +02:00
Flavio Ceolin
f6f951cc17 kernel: Fix 10.4 violations
Both operands of an operator in which the usual arithmetic
conversions are performed shall have the same essential
type category.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-04-10 09:59:37 -04:00
Anas Nashif
3f4f3f6c43 kernel: make tests of a value against zero should be made explicit
Tests of a value against zero should be made explicit, unless the
operand is effectively Boolean. This is based on MISRA rule 14.4.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-04-01 05:34:17 -04:00
Anas Nashif
25c87db860 kernel/arch: cleanup function definitions
make identifiers used in the declaration and definition identical. This
is based on MISRA rule 8.3.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-04-01 05:34:17 -04:00
Anas Nashif
bbbc38ba8f kernel: Make both operands of operators of same essential type category
Add a 'U' suffix to values when computing and comparing against
unsigned variables and other related fixes of the same MISRA rule (10.4)

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-04-01 05:34:17 -04:00
Katsuhiro Suzuki
19db485737 kernel: arch: use ENOTSUP instead of ENOSYS in k_float_disable()
This patch replaces ENOSYS into ENOTSUP to keep consistency with
the return value specification of k_float_enable().

Signed-off-by: Katsuhiro Suzuki <katsuhiro@katsuster.net>
2021-03-25 14:13:23 +01:00
Katsuhiro Suzuki
59903e2934 kernel: arch: introduce k_float_enable()
This patch introduce new API to enable FPU of thread. This is pair of
existed k_float_disable() API. And also add empty arch_float_enable()
into each architectures that have arch_float_disable(). The arc and
riscv already implemented arch_float_enable() so I do not touch
these implementations.

Motivation: Current Zephyr implementation does not allow to use FPU
on main and other system threads like as work queue. Users need to
create an other thread with K_FP_REGS for floating point programs.
Users can use FPU more easily if they can enable FPU on running
threads.

Signed-off-by: Katsuhiro Suzuki <katsuhiro@katsuster.net>
2021-03-25 14:13:23 +01:00
James Harris
33c9be90cc kernel: fix TOCTTOU issue in k_thread_name_set
Previously, a racing write to the provided string could result
in up to CONFIG_THREAD_MAX_NAME_LEN-2 bytes after the end
of user-accessible memory being leaked into the thread name.

For now, make a temporary copy. In an ideal world this could
copy directly from userspace into the thread name, but that
violates the current vrfy / impl split.

Signed-off-by: James Harris <james.harris@intel.com>
2021-03-08 19:27:23 -05:00
Andy Ross
6fb6d3cfbe kernel: Add new k_thread_abort()/k_thread_join()
Add a newer, much smaller and simpler implementation of abort and
join.  No need to involve the idle thread.  No need for a special code
path for self-abort.  Joining a thread and waiting for an aborting one
to terminate elsewhere share an implementation.  All work in both
calls happens under a single locked path with no unexpected
synchronization points.

This fixes a bug with the current implementation where the action of
z_sched_single_abort() was nonatomic, releasing the lock internally at
a point where the thread to be aborted could self-abort and confuse
the state such that it failed to abort at all.

Note that the arm32 and native_posix architectures, which have their
own thread abort implementations, now see a much simplified
"z_thread_abort()" internal API.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-02-24 16:39:15 -05:00
Andy Ross
c0c8cb0e97 kernel: Remove abort and join implementation (UNBISECTABLE)
THIS COMMIT DELIBERATELY BREAKS BISECTABILITY FOR EASE OF REVIEW.
SKIP IF YOU LAND HERE.

Remove the existing implementatoin of k_thread_abort(),
k_thread_join(), and the attendant facilities in the thread subsystem
and idle thread that support them.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-02-24 16:39:15 -05:00
Andy Ross
8e16012ab7 kernel/thread: Initialize pended_on field of struct thread_base
This got missed, leaving garbage there for restarted threads to trip
on.  Actually I see multiple uninitialized fields, which seems odd.
This code deserves some rework, thread initialization isn't a
performance path and we should probably be zeroing the struct out.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-02-24 16:39:15 -05:00
Anas Nashif
ecdc770c9b kernel: thread: do not assert on tests
remove assert that prevented us from testing non-userspace code on
platforms that can do userspace.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-02-22 14:36:06 -05:00
Anas Nashif
39f632e7f0 kernel: fix usage of KERNEL_COHERENCE macro
Add missing CONFIG_ to KERNEL_COHERENCE usage in code.

Fixes #30380

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-02-03 10:42:04 -05:00
Daniel Leung
d1495e98e2 kernel: fix arch_mem_coherent() call in spinlock
The call to arch_mem_coherent() inside spinlock.h
when spinlock validation and memory coherence enabled
is causing build error as spinlock.h does not include
kernel_arch_func.h directly. However, simply including
that file does not work either as this creates
the chicken-or-egg in the chain of include files.
In order to make spin validation work with kernel
coherence enabled, a separate function is created
to break the circular dependencies of include files.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-02-03 10:42:04 -05:00
Peter Bigot
0ab314f705 kernel: const-qualify objects used to calculate delay values
The internal API to measure time until a delay expires does not modify
the referenced timeout.  Make the functions that call it take pointers
to const objects, so that they can be used with pointer to
const-qualified containers.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2021-01-22 08:05:26 -06:00
Andy Ross
e956639dd6 kernel: Remove CONFIG_LEGACY_TIMEOUT_API
This was a fallback for an API change several versions ago.  It's time
for it to go.

Fixes: #30893

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-01-14 21:33:16 -05:00
Enjia Mai
00d7f2cfb6 kernel: thread: make offload_sem visible for releasing it outside
In order to release irq_offload semaphore outside kernel/thread.c, we
make it visible by modifying it non-static under ztest. This would be
needed such as when call irq_offload() to enter interrupt context and
a fatal error happened, then you have to release it in your fatal
handler, or the irq_offload will still be locked and no longer be
using again.

Signed-off-by: Enjia Mai <enjiax.mai@intel.com>
2020-12-09 21:52:09 -05:00
Anas Nashif
87ddddae52 Revert "kernel: fix usage of KERNEL_COHERENCE macro"
This reverts commit 67c5e6b0c0.

This is causing build issues on some platforms. Revert for now.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-08 14:27:27 -05:00
Anas Nashif
67c5e6b0c0 kernel: fix usage of KERNEL_COHERENCE macro
Add missing CONFIG_ to KERNEL_COHERENCE usage in code.

Fixes #30380

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-08 09:30:02 -05:00
Krzysztof Chruscinski
3ed8083dc1 kernel: Cleanup logger setup in kernel files
Most of kernel files where declaring os module without providing
log level. Because of that default log level was used instead of
CONFIG_KERNEL_LOG_LEVEL.

Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
2020-11-27 09:56:34 -05:00
Daniel Leung
fd7a68dbe9 kernel: use timing functions to gather thread runtime stats
This uses the timing functions to gather execution cycles of
threads. This provides greater details if arch/SoC/board
uses timer with higher resolution.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-11-11 23:55:49 -05:00
Daniel Leung
11e6b43090 tracing: roll thread switch in/out into thread stats functions
Since the tracing of thread being switched in/out has the same
instrumentation points, we can roll the tracing function calls
into the one for thread stats gathering functions.
This avoids duplicating code to call another function.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-11-11 23:55:49 -05:00
Daniel Leung
fc577c4bd1 kernel: gather basic thread runtime statistics
This adds the bits to gather the first thread runtime statictic:
thread execution time. It provides a rough idea of how much time
a thread is spent in active execution. Currently it is not being
used, pending following commits where it combines with the trace
points on context switch as they instrument the same locations.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-11-11 23:55:49 -05:00
Andrew Boie
0e30c6a0fa kernel: wipe TLS when dropping to user mode
For threads that run in supervisor mode for some time before
synchronously dropping to user mode, re-initialize the TLS
area to prevent leakage of potentially sensitive information.

We did this already for CONFIG_THREAD_USERSPACE_LOCAL_DATA
but not the new CONFIG_THREAD_LOCAL_STORAGE.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-10-26 13:06:16 +01:00
Daniel Leung
62cf1960ad kernel: support using thread local storage for errno
This enables storing errno in the thread local storage area.
With this enabled, a syscall to access errno can be avoided
when userspace is also enabled.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-10-24 10:52:00 -07:00
Daniel Leung
02b20351cd kernel: add common bits to support TLS
This adds the common struct fields and functions to support
the implementation of thread local storage in individual
architecture. This uses the thread stack to store TLS data.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-10-24 10:52:00 -07:00
Andy Ross
a8d5437799 soc/xtensa: Misc. checkpatch fixups
Code style fixes.  Kept separate from the original changes to permit
easier rebasing.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2020-10-21 06:38:53 -04:00
Andy Ross
f6d32ab0a4 kernel: Add cache coherence management framework
Zephyr SMP kernels need to be able to run on architectures with
incoherent caches.  Naive implementation of synchronization on such
architectures requires extensive cache flushing (e.g. flush+invalidate
everything on every spin lock operation, flush on every unlock!) and
is a performance problem.

Instead, many of these systems will have access to separate "coherent"
(usually uncached) and "incoherent" regions of memory.  Where this is
available, place all writable data sections by default into the
coherent region.  An "__incoherent" attribute flag is defined for data
regions that are known to be CPU-local and which should use the cache.
By default, this is used for stack memory.

Stack memory will be incoherent by default, as by definition it is
local to its current thread.  This requires special cache management
on context switch, so an arch API has been added for that.

Also, when enabled, add assertions to strategic places to ensure that
shared kernel data is indeed coherent.  We check thread objects, the
_kernel struct, waitq's, timeouts and spinlocks.  In practice almost
all kernel synchronization is built on top of these structures, and
any shared data structs will contain at least one of them.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-10-21 06:38:53 -04:00
Andrew Boie
b5a71f74a8 userspace: remove threads from domain on abort
When threads exited we were leaving dangling references to
them in the domain's mem_domain_q.

z_thread_single_abort() now calls into the memory domain
code via z_mem_domain_exit_thread() to take it off.

The thread setup code now invokes z_mem_domain_init_thread(),
avoiding extra checks in k_mem_domain_add_thread(), we know
the object isn't currently a member of a doamin.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-10-20 09:37:49 -07:00
David Komel
211e5b73c0 kernel: k_busy_wait() should return immediately on zero timeout
When CONFIG_ARCH_HAS_CUSTOM_BUSY_WAIT is not defined, cycles_to_wait
is calculated using a division operation. This calculation could take a
significant amount of time (a few microseconds on some architectures,
depending on the system clock).
In the special case of zero usec_to_wait, the function should return
immediately rather than spend time on calculations.
For example, in spi driver (spi_context.h, _spi_context_cs_control()),
k_busy_wait() can be called with zero delay. This can increase spi
transaction time significantly.
Another improvement, is moving the start_cycles initialization
before cycles_to_wait calculation, so the time it takes to calculate
cycles_to_wait will be taken into account.

Signed-off-by: David Komel <a8961713@gmail.com>
2020-10-14 14:52:06 -04:00
Aastha Grover
83b9f69755 code-guideline: Fixing code violation 10.4 Rule
Both operands of an operator in the arithmetic conversions
performed shall have the same essential type category.

Changes are related to converting the integer constants to the
unsigned integer constants

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2020-10-01 17:13:29 -04:00
Andrew Boie
e0ca403f4c kernel: add assert for mis-used k_thread_create()
k_thread_create() works as expected on both uninitialized memory,
or threads that have completely exited.

However, horrible and difficult to comprehend things can happen if a
thread object is already being used by the kernel and
k_thread_create() is called on it.

Historically this has been a problem with test cases trying to be
parsimonious with thread objects and not properly cleaning up
after themselves. Add an assertion for this which should catch
both the illegal creation of a thread already active, or threads
racing to create the same thread object.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-09-30 14:11:59 -04:00
Tomasz Bursztyka
ae5a761f85 kernel: Apply IRQ offload API change
Switching to constant parameter.

Fixes #27399

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2020-09-02 13:48:13 +02:00
Andrew Boie
9bfc8d82d0 userspace: introduce default memory domain
We make a policy change here: all threads are members of a
memory domain, never NULL. We introduce a default memory domain
for threads that haven't been assigned to or inherited another one.

Primary motivation for this change is better MMU support, as
one common configuration will be to maintain page tables at
the memory domain level.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-08-26 20:35:02 -04:00
Andrew Boie
8ce260d8df kernel: introduce supervisor-only stacks
These stacks are appropriate for threads that run purely in
supervisor mode, and also as stacks for interrupt and exception
handling.

Two new arch defines are introduced:

- ARCH_KERNEL_STACK_GUARD_SIZE
- ARCH_KERNEL_STACK_OBJ_ALIGN

New public declaration macros:

- K_KERNEL_STACK_RESERVED
- K_KERNEL_STACK_EXTERN
- K_KERNEL_STACK_DEFINE
- K_KERNEL_STACK_ARRAY_DEFINE
- K_KERNEL_STACK_MEMBER
- K_KERNEL_STACK_SIZEOF

If user mode is not enabled, K_KERNEL_STACK_* and K_THREAD_STACK_*
are equivalent.

Separately generated privilege elevation stacks are now declared
like kernel stacks, removing the need for K_PRIVILEGE_STACK_ALIGN.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-07-30 21:11:14 -04:00
Andrew Boie
e4cc84a537 kernel: update arch_switch_to_main_thread()
This now takes a stack pointer as an argument with TLS
and random offsets accounted for properly.

Based on #24467 authored by Flavio Ceolin.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-07-30 21:11:14 -04:00
Andrew Boie
b0c155f3ca kernel: overhaul stack specification
The core kernel computes the initial stack pointer
for a thread, properly aligning it and subtracting out
any random offsets or thread-local storage areas.
arch_new_thread() no longer needs to make any calculations,
an initial stack frame may be placed at the bounds of
the new 'stack_ptr' parameter passed in. This parameter
replaces 'stack_size'.

thread->stack_info is now set before arch_new_thread()
is invoked, z_new_thread_init() has been removed.
The values populated may need to be adjusted on arches
which carve-out MPU guard space from the actual stack
buffer.

thread->stack_info now has a new member 'delta' which
indicates any offset applied for TLS or random offset.
It's used so the calculations don't need to be repeated
if the thread later drops to user mode.

CONFIG_INIT_STACKS logic is now performed inside
z_setup_new_thread(), before arch_new_thread() is called.

thread->stack_info is now defined as the canonical
user-accessible area within the stack object, including
random offsets and TLS. It will never include any
carved-out memory for MPU guards and must be updated at
runtime if guards are removed.

Available stack space is now optimized. Some arches may
need to significantly round up the buffer size to account
for page-level granularity or MPU power-of-two requirements.
This space is now accounted for and used by virtue of
the Z_THREAD_STACK_SIZE_ADJUST() call in z_setup_new_thread.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-07-30 21:11:14 -04:00
Andrew Boie
62eb7d99dc arch_interface: remove unnecessary params
arch_new_thread() passes along the thread priority and option
flags, but these are already initialized in thread->base and
can be accessed there if needed.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-07-30 21:11:14 -04:00
Flavio Ceolin
c4f7faea10 random: Include header where it is used
Unit tests were failing to build because random header was included by
kernel_includes.h. The problem is that rand32.h includes a generated
file that is either not generated or not included when building unit
tests. Also, it is better to limit the scope of this file to where it is
used.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2020-07-08 21:05:36 -04:00
Anas Nashif
8aa3dc340d kernel: cleanup header inclusion
Remove unused header inclusion in kernel code.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-06-25 16:12:36 -05:00
Kumar Gala
a1b77fd589 zephyr: replace zephyr integer types with C99 types
git grep -l 'u\(8\|16\|32\|64\)_t' | \
		xargs sed -i "s/u\(8\|16\|32\|64\)_t/uint\1_t/g"
	git grep -l 's\(8\|16\|32\|64\)_t' | \
		xargs sed -i "s/s\(8\|16\|32\|64\)_t/int\1_t/g"

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-06-08 08:23:57 -05:00
Andrew Boie
378024c510 userspace: add z_is_in_user_syscall()
Certain types of system call validation may need to be pushed
deeper in the implementation and not performed in the verification
function. If such checks are only pertinent when the caller was
from user mode, we need an API to detect this situation.

This is implemented by having thread->syscall_frame be non-NULL
only while a user system call is in progress. The template for the
system call marshalling functions is changed to clear this value
on exit.

A test is added to prove that this works.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-06-03 22:33:32 +02:00
Andrew Boie
743ff98f51 kernel: wipe TLS before dropping to user mode
Ensures that TLS from when the thread was in supervisor mode
is erased, rather than rely on the arch code to do it.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-05-13 22:02:48 +02:00
Stephanos Ioannidis
aaf93205bb kconfig: Rename CONFIG_FP_SHARING to CONFIG_FPU_SHARING
This commit renames the Kconfig `FP_SHARING` symbol to `FPU_SHARING`,
since this symbol specifically refers to the hardware FPU sharing
support by means of FPU context preservation, and the "FP" prefix is
not fully descriptive of that; leaving room for ambiguity.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2020-05-08 10:58:33 +02:00
Joakim Andersson
aac2220d29 kernel: Revert: "Add static threads to k_thread_foreach functions"
Revert commit mistakenly iterating over static threads in
k_thread_foreach functions. The static threads where already included
in the for-loop, and is now duplicated.

This reverts commit bd3b4b0caf.

Signed-off-by: Joakim Andersson <joakim.andersson@nordicsemi.no>
2020-04-27 19:04:02 +02:00
Stephanos Ioannidis
0e6ede8929 kconfig: Rename CONFIG_FLOAT to CONFIG_FPU
This commit renames the Kconfig `FLOAT` symbol to `FPU`, since this
symbol only indicates that the hardware Floating Point Unit (FPU) is
used and does not imply and/or indicate the general availability of
toolchain-level floating point support (i.e. this symbol is not
selected when building for an FPU-less platform that supports floating
point operations through the toolchain-provided software floating point
library).

Moreover, given that the symbol that indicates the availability of FPU
is named `CPU_HAS_FPU`, it only makes sense to use "FPU" in the name of
the symbol that enables the FPU.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2020-04-27 19:03:44 +02:00
Andrew Boie
618426d6e7 kernel: add Z_STACK_PTR_ALIGN ARCH_STACK_PTR_ALIGN
This operation is formally defined as rounding down a potential
stack pointer value to meet CPU and ABI requirments.

This was previously defined ad-hoc as STACK_ROUND_DOWN().

A new architecture constant ARCH_STACK_PTR_ALIGN is added.
Z_STACK_PTR_ALIGN() is defined in terms of it. This used to
be inconsistently specified as STACK_ALIGN or STACK_PTR_ALIGN;
in the latter case, STACK_ALIGN meant something else, typically
a required alignment for the base of a stack buffer.

STACK_ROUND_UP() only used in practice by Risc-V, delete
elsewhere.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-04-21 18:45:45 -04:00
Andrew Boie
1f6f977f05 kernel: centralize new thread priority check
This was being done inconsistently in arch_new_thread(), just
move to the core kernel.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-04-21 18:45:45 -04:00
Andrew Boie
c0df99cc77 kernel: reduce scope of z_new_thread_init()
The core kernel z_setup_new_thread() calls into arch_new_thread(),
which calls back into the core kernel via z_new_thread_init().

Move everything that doesn't have to be in z_new_thread_init() to
z_setup_new_thread() and convert to an inline function.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-04-21 18:45:45 -04:00
Andy Ross
5a5d3daf6f kernel/timeout: Add timeout remaining/expires APIs
Add tick-based (i.e. precision resistant) inspection APIs for kernel
timeouts visible via k_timer, k_delayed work and thread timeouts
(i.e. pended/sleeping threads).  These are each available in
"remaining" and "expires" variants returning time values relative to
current time and system start.  All have system calls where applicable
(i.e. everywhere but k_delayed_work, which is not a userspace API)

The pre-existing millisecond "remaining_get()" predicates for timer
and delayed work remain, but are expressed in terms of the newer
calls.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2020-03-31 19:40:47 -04:00
Andy Ross
7832738ae9 kernel/timeout: Make timeout arguments an opaque type
Add a k_timeout_t type, and use it everywhere that kernel API
functions were accepting a millisecond timeout argument.  Instead of
forcing milliseconds everywhere (which are often not integrally
representable as system ticks), do the conversion to ticks at the
point where the timeout is created.  This avoids an extra unit
conversion in some application code, and allows us to express the
timeout in units other than milliseconds to achieve greater precision.

The existing K_MSEC() et. al. macros now return initializers for a
k_timeout_t.

The K_NO_WAIT and K_FOREVER constants have now become k_timeout_t
values, which means they cannot be operated on as integers.
Applications which have their own APIs that need to inspect these
vs. user-provided timeouts can now use a K_TIMEOUT_EQ() predicate to
test for equality.

Timer drivers, which receive an integer tick count in ther
z_clock_set_timeout() functions, now use the integer-valued
K_TICKS_FOREVER constant instead of K_FOREVER.

For the initial release, to preserve source compatibility, a
CONFIG_LEGACY_TIMEOUT_API kconfig is provided.  When true, the
k_timeout_t will remain a compatible 32 bit value that will work with
any legacy Zephyr application.

Some subsystems present timeout (or timeout-like) values to their own
users as APIs that would re-use the kernel's own constants and
conventions.  These will require some minor design work to adapt to
the new scheme (in most cases just using k_timeout_t directly in their
own API), and they have not been changed in this patch, instead
selecting CONFIG_LEGACY_TIMEOUT_API via kconfig.  These subsystems
include: CAN Bus, the Microbit display driver, I2S, LoRa modem
drivers, the UART Async API, Video hardware drivers, the console
subsystem, and the network buffer abstraction.

k_sleep() now takes a k_timeout_t argument, with a k_msleep() variant
provided that works identically to the original API.

Most of the changes here are just type/configuration management and
documentation, but there are logic changes in mempool, where a loop
that used a timeout numerically has been reworked using a new
z_timeout_end_calc() predicate.  Also in queue.c, a (when POLL was
enabled) a similar loop was needlessly used to try to retry the
k_poll() call after a spurious failure.  But k_poll() does not fail
spuriously, so the loop was removed.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2020-03-31 19:40:47 -04:00
Andrew Boie
a4c9190649 kernel: fix oops policy for k_thread_abort()
Don't generate a Z_OOPS() if k_thread_abort() is called on a
thread that isn't running. Just return to the caller instead,
much like how k_thread_join() functions.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-03-25 10:23:12 -07:00
Andrew Boie
28be793cb6 kernel: delete separate logic for priv stacks
This never needed to be put in a separate gperf table.
Privilege mode stacks can be generated by the main
gen_kobject_list.py logic, which we do here.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-03-17 20:11:27 +02:00
Andrew Boie
2dc2ecfb60 kernel: rename struct _k_object
Private type, internal to the kernel, not directly associated
with any k_object_* APIs. Is the return value of z_object_find().
Rename to struct z_object.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-03-17 20:11:27 +02:00
Andrew Boie
2f3a89fa8d kernel: rename _k_object_assignment
Private structure, rename to z_object_assignment

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-03-17 20:11:27 +02:00
Andrew Boie
4bad34e749 kernel: rename _k_thread_stack_element
Private data type, prefix with z_.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-03-17 20:11:27 +02:00
Andrew Boie
f2734ab022 kernel: use a union for kobject data values
Rather than stuffing various values in a uintptr_t based on
type using casts, use a union for this instead.

No functional difference, but the semantics of the data member
are now much clearer to the casual observer since it is now
formally defined by this union.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-03-17 20:11:27 +02:00
Andrew Boie
322816eada kernel: add k_thread_join()
Callers will go to sleep until the thread exits, either normally
or crashing out.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-03-13 08:42:43 -04:00
Joakim Andersson
bd3b4b0caf kernel: Add static threads to k_thread_foreach functions
Add iterating over the static threads for k_thread_foreach and
k_thread_foreach_unlocked iterator functions

Signed-off-by: Joakim Andersson <joakim.andersson@nordicsemi.no>
2020-03-12 10:48:29 +02:00
Andrew Boie
c4fbc08347 kernel: fixup thread monitor locking
The lock in kernel/thread.c was pulling double-duty, protecting
both the thread monitor linked list and also serializing access
to k_thread_suspend/resume functions.

The monitor list now has its own dedicated lock.

The object tracing test has been updated to use k_thread_foreach().

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-03-10 16:09:24 -04:00
Andrew Boie
6cf496f324 kernel: use sched lock for k_thread_suspend/resume
This logic should be using the sched_lock and not its own
separate lock for these two functions.

Some simplications were made; z_thread_single_resume and
z_thread_single_suspend were only used in one place, and there was
some redundant logic for whether to reschedule in the suspend case.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-03-10 09:57:58 -04:00
Andy Ross
a2f6826f9c kernel/thread: Don't clobber arch initialization of switch_handle
The recent synchronization work required that the kernel guarantee
switch_handle is non-null, but it did it in a way that works for ARC
and x86_64 but would clobber the work xtensa had already done to
populate that field.

There's no point: just make this an assert, as it's always been the
arch layer's job.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2020-02-19 08:29:35 -05:00
Andrew Boie
efc5fe07a2 kernel: overhaul unused stack measurement
The existing stack_analyze APIs had some problems:

1. Not properly namespaced
2. Accepted the stack object as a parameter, yet the stack object
   does not contain the necessary information to get the associated
   buffer region, the thread object is needed for this
3. Caused a crash on certain platforms that do not allow inspection
   of unused stack space for the currently running thread
4. No user mode access
5. Separately passed in thread name

We deprecate these functions and add a new API
k_thread_stack_space_get() which addresses all of these issues.

A helper API log_stack_usage() also added which resembles
STACK_ANALYZE() in functionality.

Fixes: #17852

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-02-08 10:02:35 +02:00