We are setting CONFIG_GEN_PRIV_STACKS when AArch64 actually uses a
statically allocated privileged stack.
This error was not captured by the tests because we only verify whether
a read/write to a privileged stack is failing, but it can fail for a lot
of reasons including when the pointer to the privileged stack is not
initialized at all, like in this case.
With this patch we deselect CONFIG_GEN_PRIV_STACKS and we fix the
mem_protect/userspace test to correctly probe the privileged stack.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Add initial support for the Cortex-M55 Core which is an implementation
of the Armv8.1-M mainline architecture and includes support for the
M‑profile Vector Extension (MVE).
The support is based on the Cortex-M33 support that already exists in
Zephyr.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
This is another API that is being used in all timer drivers and is not
internal to the clock subsystem. Remove the leading z_ and make promote
it to a cross-subsystem API.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Introduce the necessary routines to have the user thread stack correctly
mapped and the functions to swap page tables on context switch.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
There's no need to duplicate the linker section for each architecture.
Instead, move the section declaration to common-rom.ld.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Introduce the basic support code for memory domains. To each domain
is associated a top page table which is a copy of the global kernel
one. When a partition is added, corresponding memory range is made
private before its mapping is adjusted.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Can only be written at the highest Exception level implemented.
For example, if EL3 is the highest implemented Exception level,
CNTFRQ_EL0 can only be written at EL3.
Also move z_arm64_el_highest_plat_init to be called when is_el_highest
Signed-off-by: Peng Fan <peng.fan@nxp.com>
This patch adds the code managing the syscalls. The privileged stack
is setup before jumping into the real syscall.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The arch_is_user_context() function is relying on the content of the
tpidrro_el0 register to determine whether we are in user context or not.
This register is set to '1' when in EL1 and set back to '0' when user
threads are running in userspace.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
There is no strict reason to use assembly for the reset routine. Move as
much code as possible to C code using the proper helpers.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Introduce C helpers for common assembly operations and preprocessor
constructors to create helper functions to read / write system
registers.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The name for registers and bit-field in the cpu.h file is incoherent and
messy. Refactor the whole file using the proper suffixes for bits,
shifts and masks.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Remove unused symbols and related sections from the linker script. In
particular CONFIG_IS_BOOTLOADER and CONFIG_CODE_DATA_RELOCATION are not
currently supported on AArch64.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The funky order was necessary when registers were stored using a
sequence of `stp x(n), x(n+1), [sp, #-16]!`. Now that absolute offsets
are being used, we can restore natural ordering.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The most common secure monitor firmware in the ARM world is TF-A. The
current release allows up to 8 64-bit values to be returned from a
SMC64 call from AArch64 state.
Extend the number of possible return values from 4 to 8.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Instead of relying on hardcoded offset in the assembly code, introduce
the offset macros to make the code more clear.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The MT_OVERWRITE case is much more common. Redefine that flag as
MT_NO_OVERWRITE instead for those fewer cases where it is needed.
One such case is platform provided mappings. Apply them after the
common kernel mappings and use the MT_NO_OVERWRITE on them.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
There is no real reason for keeping page tables into separate pools.
Make it global which allows for more efficient memory usage and
simplifies the code.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Introduce a remove_map() to ... remove a mapping.
Add a use count to the page table pool so pages can be dynamically
allocated, deallocated and reused.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Quoting from the reference manual "Writes to PSTATE.{PAN, D, A, I, F}
occur in program order without the need for additional synchronization."
so we can remove the ISB for arch_irq_lock() and arch_irq_unlock().
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
This include make possible to use the arm_arch_timer on
platform such as Cortex-A9 or Cortex-R7 which has support for
ARM Global Timer.
The global timer is a 64 bit incrementing counter, memory-mapped
in the private memory region.
Signed-off-by: Julien Massot <julien.massot@iot.bzh>
Add support for backup SRAM initialization found in multiple STM32
microcontrollers. Linker script facilities are also provided to make it
easy to define variables in the backup SRAM.
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
The linker script defines `z_mapped_size` as follows:
```
z_mapped_size = z_mapped_end - z_mapped_start;
```
This is done with the belief that precomputed values at link time will
make the code smaller and faster.
On Aarch64, symbol values are relocated and loaded relative to the PC
as those are normally meant to be memory addresses.
Now if you have e.g. `CONFIG_SRAM_BASE_ADDRESS=0x2000000000` then
`z_mapped_size` might still have a reasonable value, say 0x59334.
But, when interpreted as an address, that's very very far from the PC
whose value is in the neighborhood of 0x2000000000. That overflows the
4GB relocation range:
```
kernel/libkernel.a(mmu.c.obj): in function `z_mem_manage_init':
kernel/mmu.c:527:(.text.z_mem_manage_init+0x1c):
relocation truncated to fit: R_AARCH64_ADR_PREL_PG_HI21
```
The solution is to define `Z_KERNEL_VIRT_SIZE` in terms of
`z_mapped_end - z_mapped_start` at the source code level. Given this
is used within loops that already start with `z_mapped_start` anyway,
the compiler is smart enough to combine the two occurrences and
dispense with a size counter, making the code effectively
slightly better for all while avoiding the Aarch64 relocation
overflow:
```
text data bss dec hex filename
1216 8 294936 296160 484e0 mmu.c.obj.arm64.before
1212 8 294936 296156 484dc mmu.c.obj.arm64.after
1110 8 9244 10362 287a mmu.c.obj.x86-64.before
1106 8 9244 10358 2876 mmu.c.obj.x86-64.after
```
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Introduce an additional single-bit flag to hold
the current length of the MPU Stack Guard size
(long or default) for the case when building
with MPU_STACK_GUARD and FPU_SHARING, in Cortex-M
platforms.
In addition, the commit adds some documentation
about the existing status flags in Arm's
thread arch.mode variable.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Major changes:
- move related functions together
- optimize add_map() not to walk the page tables *twice* on
every loop
- properly handle leftover size when a range is already mapped
- don't overwrite existing mappings by default
- return an error when the mapping fails
and make the code clearer overall.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Both _IRQ_VECTOR_TABLE_SECTION_NAME and _SW_ISR_TABLE_SECTION_NAME
are defined with asterisk at the end in an attempt to include
all related symbols in the linker script. However, these two
macros are also being used in the source code to specify
the destination sections for variables. Asterisks in the name
results in older GCC (4.x) complaining about those asterisks.
So create new macros for use in linker script, and keep
the names asterisk free.
Fixes#29936
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
With classic volatile pointer access gcc something generates
access instructions with immediate offset value, like
str w4, [x1], #4
Such instructions produce invalid syndrome in HSR register when are
trapped by hypervisor. This leads to inability to emulate device access
in hypervisor.
So we need to make sure that any access to device memory is done
with plain str/ldr instructions without offset.
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Image header is compatible with Linux aarch64 boot protocol,
so zephyr can be booted with U-boot or Xen loader.
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Add linker symbols corresponding to the start and end of the
mapped Zephyr image. This is not used by the ARM arch yet, but
is required to compile the core kernel MMU code.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Add linker symbols corresponding to the start and end of the
mapped Zephyr image. This is not used by the ARM arch yet, but
is required to compile the core kernel MMU code.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The inline asm code was not conveying in all cases that registers r1-r3
would get clobbered by the SVC handler code. In the cases that we can't
list r1-r3 in the clobber list the registers need to show up as outputs
to know that they values are not preserved by the callee.
Fixes#30393
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Firmware implementing the PSCI functions described in ARM document
number ARM DEN 0022A ("Power State Coordination Interface System
Software on ARM processors") can be used by Zephyr to initiate various
CPU-centric power operations.
It is needed for virtualization, it is used to coordinate OSes and
hypervisors and it provides the functions used for SMP bring-up such as
CPU_ON and CPU_OFF.
A new PSCI driver is introduced to setup a proper subsystem used to
communicate with the PSCI firmware, implementing the basic operations:
get_version, cpu_on, cpu_off and affinity_info.
The current implementation only supports PSCI 0.2 and PSCI 1.0
The PSCI conduit (SMC or HVC) is setup reading the corresponding
property in the DTS node.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Adds a linker section for Cortex-M instruction tightly coupled memory
(ITCM), similar to the existing section for DTCM. A new executable MPU
region is not added as there isn't currently a need to make this section
accessible to user mode. This section can be enabled by setting a device
tree chosen node zephyr,itcm.
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
Adds missing DTCM_ADDR and CCM_ADDR macros needed by the linker script
generated by scripts/gen_relocate_app.py.
Moves associated bss_end, noinit_end, and data_end linker symbols to
account for section relocation. Without this change, the section sizes
calculated in z_bss_zero() and z_data_copy() are incorrect.
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
Before hooking up the MMU driver code to the Zephyr MMU core code it's
better to match the expected variable types of the two parts.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The MMU code is currently assuming that Zephyr only uses one single set
of page tables shared by kernel and user threads. This could possibly be
not longer true in the future when multiple set of page tables can be
present and swapped at run-time.
With this patch a new arm_mmu_ptables struct is introduced that is used
to host a buffer pointing to the memory region containing the page
tables and the helper variables used to manage the page tables. This new
struct is then used by the ARM64 MMU code instead of assuming that the
kernel page tables are the only ones present.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
According to CONFIG_ARMV8_A_NS, using MT_SECURE or MT_NS, to simplify
code change, use MT_DEFAULT_SECURE_STATE instead
Signed-off-by: Peng Fan <peng.fan@nxp.com>
As timer interrupt is level triggered, we need to mask it before leaving
ISR or it will be delivered again.
Also, Xen automatically masks timer interrupt when it injects IRQ to
a guest, so we need to unmask it again, when setting new timeout.
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Peng Fan <peng.fan@nxp.com>
- Remove SYS_ prefix
- shorten POWER_MANAGEMENT to just PM
- DEVICE_POWER_MANAGEMENT -> PM_DEVICE
and use PM_ as the prefix for all PM related Kconfigs
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
ARM64 is currently using SP_ELx as stack pointer for kernel and threads
because everything is running in EL1. If support for EL0 is required, it
is necessary to switch to use SP_EL0 instead, that is the only stack
pointer that can be accessed at all exception levels by threads.
While it is not required to keep using SP_EL0 also during the
exceptions, the current code implementation makes it easier to use the
same stack pointer as the one used by threads also during the
exceptions.
This patch moves the code from using SP_ELx to SP_EL0 and fill in the
missing entries in the vector table.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
