Besides the fact that we did not have that for the current supported
boards, that makes sense for this new, virtualized mode, that is meant
to be run on top of full-fledged x86 64 CPUs.
By having xAPIC mode access only, Jailhouse has to intercept those MMIO
reads and writes, in order to examine what they do and arbitrate if it's
safe or not (e.g. not all values are accepted to ICR register). This
means that we can't run away from having a VM-exit event for each and
every access to APIC memory region and this impacts the latency the
guest OS observes over bare metal a lot.
When in x2APIC mode, Jailhouse does not require VM-exits for MSR
accesses other that writes to the ICR register, so the latency the guest
observes is reduced to almost zero.
Here are some outputs of the the command line
$ sudo ./tools/jailhouse cell stats tiny-demo
on a Jailhouse's root cell console, for one of the Zephyr demos using
LOAPIC timers, left for a couple of seconds:
Statistics for tiny-demo cell (x2APIC root, x2APIC inmate)
COUNTER SUM PER SEC
vmexits_total 7 0
vmexits_management 3 0
vmexits_cr 2 0
vmexits_cpuid 1 0
vmexits_msr 1 0
vmexits_exception 0 0
vmexits_hypercall 0 0
vmexits_mmio 0 0
vmexits_pio 0 0
vmexits_xapic 0 0
vmexits_xsetbv 0 0
Statistics for tiny-demo cell (xAPIC root, xAPIC inmate)
COUNTER SUM PER SEC
vmexits_total 4087 40
vmexits_xapic 4080 40
vmexits_management 3 0
vmexits_cr 2 0
vmexits_cpuid 1 0
vmexits_msr 1 0
vmexits_exception 0 0
vmexits_hypercall 0 0
vmexits_mmio 0 0
vmexits_pio 0 0
vmexits_xsetbv 0 0
Statistics for tiny-demo cell (xAPIC root, x2APIC inmate)
COUNTER SUM PER SEC
vmexits_total 4087 40
vmexits_msr 4080 40
vmexits_management 3 0
vmexits_cr 2 0
vmexits_cpuid 1 0
vmexits_exception 0 0
vmexits_hypercall 0 0
vmexits_mmio 0 0
vmexits_pio 0 0
vmexits_xapic 0 0
vmexits_xsetbv 0 0
See that under x2APIC mode on both Jailhouse/root-cell and guest, the
interruptions from the hypervisor are minimal. That is not the case when
Jailhouse is on xAPIC mode, though. Note also that, as a plus, x2APIC
accesses on the guest will map to xAPIC MMIO on the hypervisor just
fine.
Signed-off-by: Gustavo Lima Chaves <gustavo.lima.chaves@intel.com>
Created structures and unions needed to enable the software to
access these tables.
Also updated the helper macros to ease the usage of the MMU page
tables.
JIRA: ZEP-2511
Signed-off-by: Adithya Baglody <adithya.nagaraj.baglody@intel.com>
Some our Zephyr tools don't like seeing UTF-8 characters, as reported in
issue #4131) so a quick scan and replace for UTF-8 characters in .rst,
.h, and Kconfig files using "file --mime-encoding" (excluding the /ext
folders) finds these files to tweak.
Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
- _arch_user_mode_enter() implemented
- _arch_is_user_context() implemented
- _new_thread() will honor K_USER option if passed in
- System call triggering macros implemented
- _thread_entry_wrapper moved and now looks for the next function to
call in EDI
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
- There's no point in building up "validity" (declared volatile for some
strange reason), just exit with false return value if any of the page
directory or page table checks don't come out as expected
- The function was returning the opposite value as its documentation
(0 on success, -EPERM on failure). Documentation updated.
- This function will only be used to verify buffers from user-space.
There's no need for a flags parameter, the only option that needs to
be passed in is whether the buffer has write permissions or not.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Helper macros to ease the usage of the MMU page table structures.
Added Macros to get Page table address and Page Table Entry
values.
Signed-off-by: Adithya Baglody <adithya.nagaraj.baglody@intel.com>
The CPU first checks the page directory entry for write
or user permissions on a particular page before looking
at the page table entry.
If a region configured all pages to be non user accessible,
and this was changed for a page within it to be accessible,
the PDE would not be updated and any access would still
return a page fault.
The least amount of runtime logic to deal with this is to
indicate at build time that some pages within a region may
be marked writable or user accessible at runtime, and to
pre-set the flags in the page directory entry accordingly.
The driving need for this is the region configuration for
kernel memory, which will have user permissions set at
runtime for stacks and user-configured memory domains.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Subsequent patches will set this guard page as unmapped,
triggering a page fault on access. If this is due to
stack overflow, a double fault will be triggered,
which we are now capable of handling with a switch to
a know good stack.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
We now create a special IA hardware task for handling
double faults. This has a known good stack so that if
the kernel tries to push stack data onto an unmapped page,
we don't triple-fault and reset the system.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This has one use-case: configuring the double-fault #DF
exception handler to do an IA task switch to a special
IA task with a known good stack, such that we can dump
diagnostic information and then panic.
Will be used for stack overflow detection in kernel mode,
as otherwise the CPU will triple-fault and reset.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
A user space buffer must be validated before required operation
can proceed. This API will check the current MMU
configuration to determine if the buffer held by the user is valid.
Jira: ZEP-2326
Signed-off-by: Adithya Baglody <adithya.nagaraj.baglody@intel.com>
This breaks too easily, for example if &some_linker_variable
is used. The names don't matter at all, use preprocessor
__COUNTER__.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
In crt0.S the MMU is initialized. It uses the statically build
page tables. Here 32-bit paging scheme is used, thereby each page
table entry maps to a 4KB page. The valid regions of the memory are
specified by SOC specific file(soc.c).
JIRA: ZEP-2099
Signed-off-by: Adithya Baglody <adithya.nagaraj.baglody@intel.com>
Macro is used to create a structure to specify the boot time
page table configuration. Needed by the gen_mmu.py script to generate
the actual page tables.
Linker script is needed for the following:
1. To place the MMU page tables at 4KByte boundary.
2. To keep the configuration structure created by
the Macro(mentioned above).
Signed-off-by: Adithya Baglody <adithya.nagaraj.baglody@intel.com>
None of this is currently necessary, the spurious interrupt
stubs and exception entry code is included in the binary just
fine. To make matters worse, some data referenced lives in the
.intList section which is completely stripped out of the binary.
If in the future we find certain essential functions are being
garbage collected when they should not be, the proper way to
mitigate this is with KEEP() directives in the linker script.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This was more or less transplanted from old Viper codebase (Zephyr's
ancestor which supported paging) and adapted to current coding style.
Change-Id: I203e631f1dcd5f2fb4e9a2fa9339fc7521c7962d
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Historically, space for struct k_thread was always carved out of the
thread's stack region. However, we want more control on where this data
will reside; in memory protection scenarios the stack may only be used
for actual stack data and nothing else.
On some platforms (particularly ARM), including kernel_arch_data.h from
the toplevel kernel.h exposes intractable circular dependency issues.
We create a new per-arch header "kernel_arch_thread.h" with very limited
scope; it only defines the three data structures necessary to instantiate
the arch-specific bits of a struct k_thread.
Change-Id: I3a55b4ed4270512e58cf671f327bb033ad7f4a4f
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Convert code to use u{8,16,32,64}_t and s{8,16,32,64}_t instead of C99
integer types. There are few places we dont convert over to the new
types because of compatiability with ext/HALs or for ease of transition
at this point. Fixup a few of the PRI formatters so we build with newlib.
Jira: ZEP-2051
Change-Id: I7d2d3697cad04f20aaa8f6e77228f502cd9c8286
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
This is a start to move away from the C99 {u}int{8,16,32,64}_t types to
Zephyr defined u{8,16,32,64}_t and s{8,16,32,64}_t. This allows Zephyr
to define the sized types in a consistent manor across all the
architectures we support and not conflict with what various compilers
and libc might do with regards to the C99 types.
We introduce <zephyr/types.h> as part of this and have it include
<stdint.h> for now until we transition all the code away from the C99
types.
We go with u{8,16,32,64}_t and s{8,16,32,64}_t as there are some
existing variables defined u8 & u16 as well as to be consistent with
Zephyr naming conventions.
Jira: ZEP-2051
Change-Id: I451fed0623b029d65866622e478225dfab2c0ca8
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
CONFIG_* usually come from Kconfig, rename variables that are locally
defined to avoid confusion about where they are set.
Change-Id: I402713e6f852907e75be4bc2b916a7d15dd5649c
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The K_<thread option> flags/options avaialble to users were hidden in
the kernel private header files: move them to include/kernel.h to
publicize them.
Also, to avoid any future confusion, rename the k_thread.execution_flags
field to user_options.
Change-Id: I65a6fd5e9e78d4ccf783f3304b607a1e6956aeac
Signed-off-by: Benjamin Walsh <walsh.benj@gmail.com>
They are internal states, not user-facing.
Also prepend an underscore since they are kernel internal symbols.
Change-Id: I53740e0d04a796ba1ccc409b5809438cdb189332
Signed-off-by: Benjamin Walsh <walsh.benj@gmail.com>
Replace the existing Apache 2.0 boilerplate header with an SPDX tag
throughout the zephyr code tree. This patch was generated via a
script run over the master branch.
Also updated doc/porting/application.rst that had a dependency on
line numbers in a literal include.
Manually updated subsys/logging/sys_log.c that had a malformed
header in the original file. Also cleanup several cases that already
had a SPDX tag and we either got a duplicate or missed updating.
Jira: ZEP-1457
Change-Id: I6131a1d4ee0e58f5b938300c2d2fc77d2e69572c
Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Some thread fields were 32-bit wide, when they are not even close to
using that full range of values. They are instead changed to 8-bit fields.
- prio can fit in one byte, limiting the priorities range to -128 to 127
- recursive scheduler locking can be limited to 255; a rollover results
most probably from a logic error
- flags are split into execution flags and thread states; 8 bits is
enough for each of them currently, with at worst two states and four
flags to spare (on x86, on other archs, there are six flags to spare)
Doing this saves 8 bytes per stack. It also sets up an incoming
enhancement when checking if the current thread is preemptible on
interrupt exit.
Change-Id: Ieb5321a5b99f99173b0605dd4a193c3bc7ddabf4
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Use least significant bits for common flags and high bits for
arch-specific ones.
Change-Id: I982719de4a24d3588c19a0d30bbe7a27d9a99f13
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Also remove some old cflags referencing directories that do not exist
anymore.
Also replace references to legacy APIs in doxygen documentation of
various functions.
Change-Id: I8fce3d1fe0f4defc44e6eb0ae09a4863e33a39db
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
nano_cpu_idle/nano_cpu_atomic_idle were not ported to the unified
kernel, and only the old APIs were available. There was no real impact
since, in the unified kernel, only the idle thread should really be
doing power management. However, with a single-threaded kernel, these
functions can be useful again.
The kernel internals now make use of these APIs instead of the legacy
ones.
Change-Id: Ie8a6396ba378d3ddda27b8dd32fa4711bf53eb36
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Also remove NO_METRIC, which is not referenced anywhere anymore.
Change-Id: Ieaedf075af070a13aa3d975fee9b6b332203bfec
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
There was a lot of duplication between architectures for the definition
of threads and the "nanokernel" guts. These have been consolidated.
Now, a common file kernel/unified/include/kernel_structs.h holds the
common definitions. Architectures provide two files to complement it:
kernel_arch_data.h and kernel_arch_func.h. The first one contains at
least the struct _thread_arch and struct _kernel_arch data structures,
as well as the struct _callee_saved and struct _caller_saved register
layouts. The second file contains anything that needs what is provided
by the common stuff in kernel_structs.h. Those two files are only meant
to be included in kernel_structs.h in very specific locations.
The thread data structure has been separated into three major parts:
common struct _thread_base and struct k_thread, and arch-specific struct
_thread_arch. The first and third ones are included in the second.
The struct s_NANO data structure has been split into two: common struct
_kernel and arch-specific struct _kernel_arch. The latter is included in
the former.
Offsets files have also changed: nano_offsets.h has been renamed
kernel_offsets.h and is still included by the arch-specific offsets.c.
Also, since the thread and kernel data structures are now made of
sub-structures, offsets have to be added to make up the full offset.
Some of these additions have been consolidated in shorter symbols,
available from kernel/unified/include/offsets_short.h, which includes an
arch-specific offsets_arch_short.h. Most of the code include
offsets_short.h now instead of offsets.h.
Change-Id: I084645cb7e6db8db69aeaaf162963fe157045d5a
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Updates x86 floating point support to reflect changes that have
been made in recent months.
* Many, many, many cosmetic changes (mostly revisions to comments).
* Elimination of unnecessary function aliases that were needed
to support the task and fiber versions of certain APIs.
* Elimination of run-time code to enable a thread's "FP regs"
option bit if the "SSE regs" option bit was set. The kernel
now recognizes that the thread is using the FPU as long as
either option bit is set. (If the thread has both option bits
enabled this is the same as if only the "SSE regs" bit is set.)
Change-Id: Ic12abc54b6fa78921749b546d8debf23e7ad232d
Signed-off-by: Allan Stephens <allan.stephens@windriver.com>
Symbols now use the K_ prefix which is now standard for the
unified kernel. Legacy support for these symbols is retained
to allow existing applications to build successfully.
Change-Id: I3ff12c96f729b535eecc940502892cbaa52526b6
Signed-off-by: Allan Stephens <allan.stephens@windriver.com>
Adds standard prefix to symbolic option that flags a thread
as essential to system operation.
Change-Id: Ia904a81ce343fdd1cd44caaaeae641d822777f9b
Signed-off-by: Allan Stephens <allan.stephens@windriver.com>
They were the same, standardize on the lowercase one.
Change-Id: I8bca080e45f3e0970697d4451e468b9081f96f5f
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
When adding a thread to the ready queue, it is often known at that time
if the thread added will be the next one to run or not. So, instead of
simply updating the ready queues and the bitmask, also cache what that
thread is, so that when the scheduler is invoked, it can simply fetch it
from there. This is only done if there is a thread in the cache, since
the way the cache is updated is by comparing the priorities of the
thread being added and the cached thread.
When a thread is removed from the ready queue, if it is currently the
cached thread, it is also removed from the cache. The cache is not
updated at this time, since this would be a preemptive fetching that
could be overriden before the newly cached thread would even be
scheduled in.
Finally, when a thread is scheduled in, it now becomes the cached thread
since the fact that it is running means that by definition it was the
next one to run.
Doing this can speed up considerably some context switch times,
especially when a thread is preempted by an interrupt and the same
thread is scheduled when the interrupt exits.
Change-Id: I6dc8391cfca566699bb9b217eafe6bc6a063c8bb
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Exception stubs now just push the handler and in some cases a dummy
error code before jumping to the exception handling code, never to
return.
Change-Id: I6a79d9243deb3fc7ccdae003dd0917364c0aa304
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Interrupt stubs now just push the ISR and parameter onto the stack
and jump to the common interrupt code, never to return.
Change-Id: I82543d8148b5c7dfe116c43f41791f852614bb28
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This header has a bunch of data structure definitions and macros useful
for manipulating segment descriptors on X86. The old IDT_ENTRY defintion
is removed in favor of the new 'struct segment_descriptor' which can be
used for all segment descriptor types and not just IRQ gates.
We also add some inline helper functions for examining segment registers,
descriptor tables, and doing far jumps/calls.
Change-Id: I640879073afa9765d2a214c3fb3c3305fef94b5e
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The '_timeout' structure is needed by dummy threads so that they can
handle timeouts.
Change-Id: Iefabd6ad93c8e176e95ce4262f5f3544dc90b7d5
Signed-off-by: Peter Mitsis <peter.mitsis@windriver.com>
