In the Armv8R AArch64 profile[1], the Armv8R AArch64 is always in secure
mode. But the FVP_BaseR_AEMv8R before version 11.16.16 doesn't strictly
follow this rule. It still has some non-secure registers
(e.g. CNTHP_CTL_EL2).
Since version 11.16.16, the FVP_BaseR_AEMv8R has fixed this issue. The
CNTHP_XXX_EL2 registers have been changed to CNTHPS_XXX_EL2. So the
FVP_BaseR_AEMv8R (version >= 11.16.16) cannot boot Zephyr. This patch
will fix it.
[1] https://developer.arm.com/documentation/ddi0600/latest/
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Change-Id: If986f34dc080ae7a8b226bba589b6fe616a4260b
This change adds `k_cycle_get_64()` on platforms that
support a 64-bit cycle counter.
The interface functions `arch_k_cycle_get_64()` and
`sys_clock_cycle_get_64()` are also introduced.
Fixes#39934
Signed-off-by: Christopher Friedt <chrisfriedt@gmail.com>
This commit adds Xen hypervisor call interface for arm64 architecture.
This is needed for further development of Xen features in Zephyr.
Signed-off-by: Dmytro Firsov <dmytro_firsov@epam.com>
When _app_smem region is empty, alignment is also needed. If there
is no alignment, the _app_smem_start used by arm mpu can be lower
than __rodata_region_end, and this two regions can overlap.
The Armv8-R aarch64 MPU does not allow overlapped regions.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Fix the attrs of init fix mem regions for userspace.
Allow RAM region and RO region be read-only for EL0.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Add a new macro MEM_DOMAIN_ALIGN_AND_SIZE for mmu and mpu mem
alignment.
MEM_DOMAIN_ALIGN_AND_SIZE is
- CONFIG_MMU_PAGE_SIZE, when mmu is enabled.
- CONFIG_ARM_MPU_REGION_MIN_ALIGN_AND_SIZE when mpu enabled.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Include the new introduced include/arch/arm64/mm.h instead of the
arm_mmu.h or arm_mpu.h.
Unify function names z_arm64_thread_pt_init/z_arm64_swap_ptables with
z_arm64_thread_mem_domains_init/z_arm64_swap_mem_domains for mmu and
mpu, because:
1. mmu and mpu have almost the same logic.
2. mpu doesn't have ptables.
3. using the function names help reducing "#if define" macros.
Similarly, change z_arm64_ptable_ipi to z_arm64_domain_sync_ipi
And fix a log bug in arm_mmu.c.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
The defination of K_MEM_PARTITION_* marcos denpens on wheather it is
mmu or mpu, so I move those definations for mmu into arm_mmu.h and
create mpu relevant ones in arm_mpu.h
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
The arm_mmu.h and arm_mpu.h have some common logic. To reduce some
redundency and the ugly 'if defined' marcos, I add a new header
include/arch/arm64/mm.h to include arm_mmu.h or arm_mpu.h depending on
CONFIG_ARM_MMU or CONFIG_ARM_MPU. Thus, in the future, the common code
of mmu and mpu should be in include/arch/arm64/mm.h.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
This patch mainly moves mpu related code from
arch/arm64/core/cortex_r/mpu/ to arch/arm64/core/cortex_r/ and moves
the mpu header files from include/arch/arm64/cortex_r/mpu/ to
include/arch/arm64/cortex_r/
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Fixes: #38591, #38207, #37861
The commit 65a2de84a9 aligned the data
linker symbol for sections and regions.
The data region symbol start has been placed outside the sections thus
being defined as the address of the region before alignment of the first
section in the data region, usually the `datas` section.
The symbol defining the start address of the data section is after
section alignment.
In most cases the address of the data region start and datas section
start will be identical, but not always.
The data region symbol is a new linker symbol and existing code has
been depending on the old data section start symbol.
Thus, the update to the use of the data region start symbol instead of
data ram start symbol thus results in a different address when the
section is aligned to a different address.
To ensure the original behavior in all cases, the data region start
address is now moved inside the data section.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Cleanup and preparation commit for linker script generator.
Zephyr linker scripts provides start and end symbols for each larger
areas in the linker script.
The symbols _image_text_start and _image_text_end sometimes includes
linker/kobject-text.ld. This mean there must be both the regular
__text_start and __text_end symbols for the pure text section, as well
as <group>_start and <group>_end symbols.
The symbols describing the text region which covers more than just the
text section itself will thus be changed to:
_image_text_start -> __text_region_start
_image_text_end -> __text_region_end
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Cleanup and preparation commit for linker script generator.
Zephyr linker scripts provides start and end symbols for each larger
areas in the linker script.
The symbols _image_rom_start and _image_rom_end corresponds to the group
ROMABLE_REGION defined in the ld linker scripts.
The symbols _image_rodata_start and _image_rodata_end is not placed as
independent group but covers common-rom.ld, thread-local-storage.ld,
kobject-rom.ld and snippets-rodata.ld.
This commit align those names and prepares for generation of groups in
linker scripts.
The symbols describing the ROMABLE_REGION will be renamed to:
_image_rom_start -> __rom_region_start
_image_rom_end -> __rom_region_end
The rodata will also use the group symbol notation as:
_image_rodata_start -> __rodata_region_start
_image_rodata_end -> __rodata_region_end
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Cleanup and preparation commit for linker script generator.
Zephyr linker scripts provides start and end symbols for each section,
and sometimes even size and LMA start symbols.
Generally, start and end symbols uses the following pattern, as:
Section name: foo
Section start symbol: __foo_start
Section end symbol: __foo_end
However, this pattern is not followed consistently.
To allow for linker script generation and ensure consistent naming of
symbols then the following pattern is introduced consistently to allow
for cleaner linker script generation.
Section name: foo
Section start symbol: __foo_start
Section end symbol: __foo_end
Section size symbol: __foo_size
Section LMA start symbol: __foo_load_start
This commit aligns the symbols for _data_ram/rom to other symbols and in
such a way they follow consistent pattern which allows for linker script
and scatter file generation.
The symbols are named according to the section name they describe.
Section name is `data`
A new group named data_region is introduced which instead spans all the
input and output sections that was previously covered by
__data_ram_start, __data_ram_end, and __data_rom_start.
The following symbols are aligned in this commit:
- __data_ram_start -> __data_region_start
- __data_ram_end -> __data_region_end
- __data_rom_start -> __data_region_load_start
The following new symbols are introduced so that the data section is
aligned with other sections:
- __data_end
- __data_start
value identical to __data_region_start but describes start of
the section.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Commit 6b3c5e8bb2 removed the use of
_DATA_IN_ROM but kept the now unused define.
This commit removes the corresponding define which is no longer used.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Move the IDT_LIST memory region to the location recommended by
`intlist.ld`. The documentation specifies that this region should not
overlap other regions, and there is no guarantee that the area after the
`SRAM` region is not used. The end of the address space is much less
likely to be a valid RAM address.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Current location options for linker source files includes init and
noinit ram data, but only a noinit ram section. This makes it impossible
for application code to define an initialized RAM output section,
such as with the Z_ITERABLE_SECTION_RAM() helpers.
Adding a DATA_SECTIONS linker source option for this use case.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Add strong definition z_arm64_el2_plat_init() and it is controlled
by CONFIG_SOC_FVP_AEMV8R_EL2_INIT.
VMPIDR_EL2 must be set manually on EL2. The purpose of VMPIDR_EL2 is
that holds the value of the Virtualization Multiprocessor ID and This
is the value returned by EL1 reads of MPIDR_EL1
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
This adds FPU sharing support with a lazy context switching algorithm.
Every thread is allowed to use FPU/SIMD registers. In fact, the compiler
may insert FPU reg accesses in anycontext to optimize even non-FP code
unless the -mgeneral-regs-only compiler flag is used, but Zephyr
currently doesn't support such a build.
It is therefore possible to do FP access in IRS as well with this patch
although IRQs are then disabled to prevent nested IRQs in such cases.
Because the thread object grows in size, some tests have to be adjusted.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Add the exception depth count to tpidrro_el0 and make it available
through the arch_exception_depth() accessor.
The IN_EL0 flag is now updated unconditionally even if userspace is
not configured. Doing otherwise made the code rather hairy and
I doubt the overhead is measurable.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Due to the use of gperf to generate hash table for kobjects,
the addresses of these kobjects cannot change during the last
few phases of linking (especially between zephyr_prebuilt.elf
and zephyr.elf). Because of this, the gperf generated data
needs to be placed at the end of memory to avoid pushing symbols
around in memory. This prevents moving these generated blocks
to earlier sections, for example, pinned data section needed
for demand paging. So create placeholders for use in
intermediate linking to reserve space for these generated blocks.
Due to uncertainty on the size of these blocks, more space is
being reserved which could result in wasted space. Though, this
retains the use of hash table for faster lookup.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The ARM64 port is currently using SP_EL0 for everything: kernel threads,
user threads and exceptions. In addition when taking an exception the
exception code is still using the thread SP without relying on any
interrupt stack.
If from one hand this makes the context switch really quick because the
thread context is already on the thread stack so we have only to save
one register (SP) for the whole context, on the other hand the major
limitation introduced by this choice is that if for some reason the
thread SP is corrupted or pointing to some unaccessible location (for
example in case of stack overflow), the exception code is unable to
recover or even deal with it.
The usual way of dealing with this kind of problems is to use a
dedicated interrupt stack on SP_EL1 when servicing the exceptions. The
real drawback of this is that, in case of context switch, all the
context must be copied from the shared interrupt stack into a
thread-specific stack or structure, so it is really slow.
We use here an hybrid approach, sacrificing a bit of stack space for a
quicker context switch. While nothing really changes for kernel threads,
for user threads we now use the privileged stack (already present to
service syscalls) as interrupt stack.
When an exception arrives the code now switches to use SP_EL1 that for
user threads is always pointing inside the privileged portion of the
stack of the current running thread. This achieves two things: (1)
isolate exceptions and syscall code to use a stack that is isolated,
privileged and not accessible to user threads and (2) the thread SP is
not touched at all during exceptions, so it can be invalid or corrupted
without any direct consequence.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Add the ability to define architecture specific structures, notably
the ability to extend struct _cpu with per-CPU arch-specific stuff that
can be accessed with _current_cpu->arch.* similarly to _current->arch.*
for per-thead architecture data.
This is opt-in for architectures that want to benefit from this,
otherwise empty defaults are provided. A placeholder for ARM64 is
included to show the pattern.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
We can find caller of z_arm64_mmu_init is on primary
core or not, so no need to check mpidr, just add a
function parameter.
Signed-off-by: Jiafei Pan <Jiafei.Pan@nxp.com>
Let's fully exploit tpidrro_el0 by storing in it the current CPU's
struct _cpu instance alongside the userspace mode flag bit. This
greatly simplifies the code needed to get at the cpu structure, and
this paves the way to much simpler multi cluster support, as there
is no longer the need to decode MPIDR all the time.
The same code is used in the !SMP case as there are benefits there too
such as avoiding the literal pool, and it looks cleaner.
The tpidrro_el0 value is no longer stored in the exception stack frame.
Instead, we simply restore the user mode flag based on the SPSR value.
This way, more flag bits could be used independently in the future.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
When MPU is enabled, the sections need to be 64 bytes aligned.
In the case of MMU, BSS section will be 4k aligned, because the first
variable in BSS section 'base_xlat_table' is explicitly aligned by
'__aligned(NUM_BASE_LEVEL_ENTRIES * sizeof(uint64_t))'.
However, with MPU, we do not have such a variable. So it's necessary
to fix the alignment of the BSS section in the linker.ld
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
According to Armv8-R64 Spec, MPU related meta data(region base/limit)
is 64 bits. So we need to re-define MPU related data structure here.
Signed-off-by: Haibo Xu <haibo.xu@arm.com>
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
If default config ARM_MMU is set to n, samples/tests will have
compilation error. This is because the arch/arm/aarch64/arm_mmu.h
is always included.
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
The structure for the arm64_cpu_init array has to carry the cache
alignment on the whole structure and not on some internal padding
to achieve the desired effect.
And align struct __esf to a 16-byte boundary which will also align
its size accordingly. This structure is allocated on the stack on
exception entry and the ABI prescribed 16-byte stack alignment
should be preserved.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Split ARM and ARM64 architectures.
Details:
- CONFIG_ARM64 is decoupled from CONFIG_ARM (not a subset anymore)
- Arch and include AArch64 files are in a dedicated directory
(arch/arm64 and include/arch/arm64)
- AArch64 boards and SoC are moved to soc/arm64 and boards/arm64
- AArch64-specific DTS files are moved to dts/arm64
- The A72 support for the bcm_vk/viper board is moved in the
boards/bcm_vk/viper directory
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
