Commit Graph

1267 Commits

Author SHA1 Message Date
Daniel Leung
605cad1190 xtensa: cache: XCC needs to declare variable outside for loop
XCC doesn't like that a for loop which declares the loop
variable inline. So extract the declaration of the loop
variable outside the for loop.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-07-22 15:41:11 +03:00
Daniel Leung
c0c0dc3d17 x86: type cast to uint8_t* for bit ops
This changes the type casting of the incoming addresses to
the bit ops from uint32_t* to uint8_t*. This avoids compilers
and static analyzers complaining about the incoming out of
bound access if incoming argument is an array smaller than
4 bytes.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-07-22 07:24:11 -05:00
Huifeng Zhang
c34960bc87 arch: arm64: Unify the initialization of MMU and MPU
Because MMU and MPU should not be enabled together and they provide
the same functionalities.

Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
2021-07-13 09:30:29 -04:00
Huifeng Zhang
b89c727c8e soc: arm64: arm: fvp_aemv8r: Init VMPIDR_EL2 in el2 plat init.
Add strong definition z_arm64_el2_plat_init() and it is controlled
by CONFIG_SOC_FVP_AEMV8R_EL2_INIT.

VMPIDR_EL2 must be set manually on EL2. The purpose of VMPIDR_EL2 is
that holds the value of the Virtualization Multiprocessor ID and This
is the value returned by EL1 reads of MPIDR_EL1

Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
2021-07-13 09:30:29 -04:00
Tim Lin
ef3c8507a6 ite: drivers/flash: add flash driver for it8xxx2
Add flash driver for it8xxx2. The driver can implement
flash read, write and erase that will be mapped to the
ram section for executing.

TEST="flash write 0x80000 0x10 0x20 0x30 0x40 ..."
     "flash read 0x80000 0x100"
     "flash erase 0x80000 0x1000"

Signed-off-by: Tim Lin <tim2.lin@ite.corp-partner.google.com>
2021-06-22 15:15:20 -04:00
Øyvind Rønningstad
382bbacb0a tfm: Put saving of FPU context into its own file so it can be reused
Also, this eases readability.

The new API can be used any time all FP registers must be manually
saved and restored for an operation.

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2021-06-07 15:23:22 +02:00
Dino Li
2bf63134e8 arch: riscv: linker script: add support for rom_start section
With this change, we can put contents into rom_start section
by calling zephyr_linker_sources(ROM_START ...)

Signed-off-by: Dino Li <Dino.Li@ite.com.tw>
Co-authored-by: Torsten Tejlmand Rasmussen
Change-Id: If1169423b013d3e4df52d91cdb2fbdddc3bace7b
2021-06-04 16:31:00 -05:00
Maksim Masalski
78ba2ec830 coding guidelines: add to function prototypes form named parameters
Function types shall be in prototype form with named parameters

Found as a coding guideline violation (MISRA R8.2) by static
coding scanning tool.

Signed-off-by: Maksim Masalski <maksim.masalski@intel.com>
2021-06-04 16:20:06 -05:00
Stephanos Ioannidis
bbbc9b132b nios2: Fix C++ exception handling info linking
The NIOS2 architecture linker script was including `cplusplus-ram.ld`
linker script after `__data_ram_end`, and this caused the content of
`.gcc_except_table` section to be not copied to the RAM by the
`z_data_copy` function; leading to the C++ exception handling
malfunction.

This commit relocates the `cplusplus-ram.ld` linker script inclusion
such that the contents of the relevant sections are properly copied by
the `z_data_copy` function.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2021-05-28 12:07:46 -05:00
Torsten Rasmussen
7d37fe21d1 cmake: c++ exceptions linking support
Fixes part of: #32448

This commit updates the CMake CMAKE_CXX_LINK_EXECUTABLE to include
crtbegin.o and crtend.o at the right locations when linking with gcc.

It also updates linker scripts to ensure proper location of the
exception header frame sections.

This ensure proper handling of exceptions for those architectures
- x86
- xtensa
- riscv32

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-05-27 07:43:28 -05:00
Anas Nashif
5b8d418058 nios2: revert back to builtin sys_io functions
Some time ago we did a cleanup of sys_io function and left nios2 broken,
especially on the MAX10 board. Revert back to the original
implementation for this architecture.

Fixes #35694

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-05-26 18:09:47 -05:00
Marek Pieta
94cc9a9bdb arch: arm: Fix build warning
Change fixes a build warning related to attribute ignored in
declaration of struct. The __packed attribute is removed.

Signed-off-by: Marek Pieta <Marek.Pieta@nordicsemi.no>
2021-05-26 11:17:45 -05:00
Daniel Leung
fb6a929f97 x86: ia32: put IRQ stubs into pinned linker section if needed
This changes the ARCH_IRQ_CONNECT() to put the IRQ stubs into
pinned linker section if needed.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-05-10 16:00:43 -05:00
Daniel Leung
512cb905d1 x86: ia32/linker: add boot and pinned sections
This adds both boot and pinned sections to the linker
script for ia32. This is required for enabling demand
paging for kernel and data.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-05-10 16:00:43 -05:00
Daniel Leung
af49ec0277 linker: remove TEXT_START macro
There is exactly one function being defined with TEXT_START
macro so the x86-32 __start can appear at the beginning of
text section. Since no one else is using it, better remove
TEXT_START to simplify things.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-05-10 16:00:43 -05:00
Evgeniy Paltsev
a70187fb31 ARC: arch.h: setup ARCH_STACK_PTR_ALIGN correctly for 64 bit
Stack is aligned by 8 for 64 bit ISA and 4 for 32 bit ISA.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
2021-05-07 14:55:49 -05:00
Evgeniy Paltsev
0d859796be ARC: make variables with regs and addresses bit agnostic
Make variables where we store CPU registers values and
memory addresses bit agnostic.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
2021-05-07 14:55:49 -05:00
Evgeniy Paltsev
ab17a59ba5 ARC: mark accesses which are 32 bit despite of platform bittnes
Mark the places where we intentionally use st instead of STR for
code common for ARCv2 and ARCv3.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
2021-05-07 14:55:49 -05:00
Evgeniy Paltsev
8cb122ea5d ARC: reuse headers for both ARCv3 and ARCv3 if possible
Reuse ARCv2 headers [where it is possible] for ARCv3.
In this commit we simply allow to use them for ARCv3, we'll
move it to proper folder and rename them [where it is required]
in the upcoming cleanup patch.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
2021-05-07 14:55:49 -05:00
Evgeniy Paltsev
fb8dc2b83e ARC: split sys-io for common and ARCv2-only parts
Split ARC sys-io.h implementation for two part:
 * AUX reg accessors - ASM based which and valid for ARCv2
 * MMIO accessors - writen in C and valid for both ARCv2 and ARCv3

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
2021-05-07 14:55:49 -05:00
Evgeniy Paltsev
1aef12d8fa ARC: add asm-compat macro for MWDT toolchain
Extend asm-compat macroses so they can be used with ARC MWDT
toolchain for ARCv2

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
2021-05-07 14:55:49 -05:00
Vineet Gupta
3d56fff7a4 ARC: add asm-compat macros
Add asm-compat macroses to be able to co create assembler code
which works and can be build for both ARCv2 and ARCv3 ISAs

Signed-off-by: Vineet Gupta <vgupta@synopsys.com>
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
2021-05-07 14:55:49 -05:00
Daniel Leung
783b20712e arch: implement brute force find_lsb_set()
On RISC-V 64-bit, GCC complains about undefined reference
to 'ffs' via __builtin_ffs(). So implement a brute force
way to do it. Once the toolchain has __builtin_ffs(),
this can be reverted.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-05-07 13:36:22 -04:00
Bradley Bolen
3b6707f140 arch: arm: aarch32: Use mpu headers from common aarch32 location
The mpu headers moved in order to support the Cortex-R mpu in later
commits.  Use the new locations.

Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
2021-05-06 19:39:09 +02:00
Bradley Bolen
2076131543 arch: arm: aarch32: mpu: Move mpu header files up one level
Prepare for Cortex-R MPU support by moving the MPU headers to the
aarch32 common directory.  Add compat headers to ease the transition for
the Cortex-M external modules.

Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
2021-05-06 19:39:09 +02:00
Gerard Marull-Paretas
280ca7a632 arch: replace power/power.h with pm/pm.h
Replace old header with the new one.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2021-05-05 18:35:49 -04:00
Øyvind Rønningstad
a2cfb8431d arch: arm: Add code for swapping threads between secure and non-secure
This adds code to swap_helper.S which does special handling of LR when
the interrupt came from secure. The LR value is stored to memory, and
put back into LR when swapping back to the relevant thread.

Also, add special handling of FP state when switching from secure to
non-secure, since we don't know whether the original non-secure thread
(which called a secure service) was using FP registers, so we always
store them, just in case.

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2021-05-05 13:00:31 +02:00
Nicolas Pitre
f1f63dda17 arm64: FPU context switching support
This adds FPU sharing support with a lazy context switching algorithm.

Every thread is allowed to use FPU/SIMD registers. In fact, the compiler
may insert FPU reg accesses in anycontext to optimize even non-FP code
unless the -mgeneral-regs-only compiler flag is used, but Zephyr
currently doesn't support such a build.

It is therefore possible to do FP access in IRS as well with this patch
although IRQs are then disabled to prevent nested IRQs in such cases.

Because the thread object grows in size, some tests have to be adjusted.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2021-05-03 11:56:50 +02:00
Nicolas Pitre
a82fff04ff arm64: implement exception depth count
Add the exception depth count to tpidrro_el0 and make it available
through the arch_exception_depth() accessor.

The IN_EL0 flag is now updated unconditionally even if userspace is
not configured. Doing otherwise made the code rather hairy and
I doubt the overhead is measurable.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2021-05-03 11:56:50 +02:00
Jordan Yates
a0909ac00b linker: stm32f0: move ram vector table to snippets
Move the STM32F0 specific ram vector table sections into a dedicated
linker snippet included by the new `SRAM_VECTOR_TABLE` symbol.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2021-04-30 15:16:02 +02:00
Jordan Yates
318ee971c5 boards: stm32f0: explicit RAM vector table control
Add a new Kconfig symbol that explicitly controls whether the vector
table should be placed in RAM. This eliminates the side effect of
`IS_BOOTLOADER` controlling vector table location. Making the condition
a positive assertion also allows the config to be used in CMakeLists
conditions (`zephyr_linker_sources_ifdef()`, etc).

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2021-04-30 15:16:02 +02:00
Daniel Leung
2bd871df2a sparc: add _image_rodata_start/end to linker script
This adds the _image_rodata_start and _image_rodata_end
symbols to SPARC's arch linker script.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-04-28 22:25:42 +02:00
Daniel Leung
b23784bf7e nios2: add _image_rodata_start/end to linker script
This adds the _image_rodata_start and _image_rodata_end
symbols to NIOS2's linker script.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-04-28 22:25:42 +02:00
Daniel Leung
1117169980 kernel: generate placeholders for kobj tables before final build
Due to the use of gperf to generate hash table for kobjects,
the addresses of these kobjects cannot change during the last
few phases of linking (especially between zephyr_prebuilt.elf
and zephyr.elf). Because of this, the gperf generated data
needs to be placed at the end of memory to avoid pushing symbols
around in memory. This prevents moving these generated blocks
to earlier sections, for example, pinned data section needed
for demand paging. So create placeholders for use in
intermediate linking to reserve space for these generated blocks.
Due to uncertainty on the size of these blocks, more space is
being reserved which could result in wasted space. Though, this
retains the use of hash table for faster lookup.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-04-27 13:32:00 -04:00
Mario Jaun
43ef9013a8 arch: arm: add define REGION_RAM_NOCACHE_ATTR
Add a define for non cachaeable RAM for MPU region definitions.

Signed-off-by: Mario Jaun <mario.jaun@gmail.com>
2021-04-27 14:16:35 +02:00
Mario Jaun
100227cc85 linker: arm: add SRAM3/4 linker section
Add a linker section for SRAM3/4 if it is enabled in the device tree.

Signed-off-by: Mario Jaun <mario.jaun@gmail.com>
2021-04-27 14:16:35 +02:00
Jiafei Pan
86a58bf7e4 aarch64: lib_helpers: add some system registers' helper
Add helper for system register: clidr_el1, csselr_el1,
ccsidr_el1.

Signed-off-by: Jiafei Pan <Jiafei.Pan@nxp.com>
2021-04-26 13:39:39 +02:00
Carlo Caione
256ca55476 arm64: Rework stack usage
The ARM64 port is currently using SP_EL0 for everything: kernel threads,
user threads and exceptions. In addition when taking an exception the
exception code is still using the thread SP without relying on any
interrupt stack.

If from one hand this makes the context switch really quick because the
thread context is already on the thread stack so we have only to save
one register (SP) for the whole context, on the other hand the major
limitation introduced by this choice is that if for some reason the
thread SP is corrupted or pointing to some unaccessible location (for
example in case of stack overflow), the exception code is unable to
recover or even deal with it.

The usual way of dealing with this kind of problems is to use a
dedicated interrupt stack on SP_EL1 when servicing the exceptions. The
real drawback of this is that, in case of context switch, all the
context must be copied from the shared interrupt stack into a
thread-specific stack or structure, so it is really slow.

We use here an hybrid approach, sacrificing a bit of stack space for a
quicker context switch. While nothing really changes for kernel threads,
for user threads we now use the privileged stack (already present to
service syscalls) as interrupt stack.

When an exception arrives the code now switches to use SP_EL1 that for
user threads is always pointing inside the privileged portion of the
stack of the current running thread. This achieves two things: (1)
isolate exceptions and syscall code to use a stack that is isolated,
privileged and not accessible to user threads and (2) the thread SP is
not touched at all during exceptions, so it can be invalid or corrupted
without any direct consequence.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2021-04-23 06:32:20 -04:00
Nicolas Pitre
f97d12936e kernel: add an architecture specific structs header
Add the ability to define architecture specific structures, notably
the ability to extend struct _cpu with per-CPU arch-specific stuff that
can be accessed with _current_cpu->arch.* similarly to _current->arch.*
for per-thead architecture data.

This is opt-in for architectures that want to benefit from this,
otherwise empty defaults are provided. A placeholder for ARM64 is
included to show the pattern.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2021-04-21 09:03:47 -04:00
Jiafei Pan
7889364771 arm64: refine the code for primary core checking
We can find caller of z_arm64_mmu_init is on primary
core or not, so no need to check mpidr, just add a
function parameter.

Signed-off-by: Jiafei Pan <Jiafei.Pan@nxp.com>
2021-04-19 11:00:05 -04:00
Jordan Yates
18d3f4dc36 linker: cc13x2_cc26x2: CCFG to linker snippets
Move the TI_CCFG linker section code into a dedicated snippet in the soc
directory.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2021-04-15 12:25:55 +02:00
Jordan Yates
8fb37b94fd linker: cc13x2_cc26x2: CCFG registers as partition
Define the FLASH_CCFG memory region from a devicetree partition instead
of from math in the linker file. Removing the special math case results
in the FLASH_CCFG region overlapping the FLASH region, but the linker
accepts this until the FLASH region actually starts placing variables
in the FLASH_CCFG region.

As a result, applications that don't fit in (FLASH_SIZE - 88) bytes will
still fail to link, just with an overlapping memory region error instead
of an overflow error.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2021-04-15 12:25:55 +02:00
Jordan Yates
a4150c0483 linker: stm32wb: move IPC RAM definition to DT
Move the definition of the two IPC RAM blocks from `#define`'s in family
linker scripts to proper devicetree nodes. Use the devicetree nodes to
generate the memory regions.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2021-04-15 12:25:55 +02:00
Jordan Yates
ddc25b4bdf linker: aarch32: simplify memory region generation
Simplify the linker script by using the helper macros introduced in
<linker/devicetree_regions.h>. The conditional checks on the Kconfig
symbols are discarded as their default values are typically set via
the status="okay" property of the nodes, and hence the behaviour doesn't
change.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2021-04-15 12:25:55 +02:00
Nicolas Pitre
88477906f0 arm64: hold curr_cpu instance in tpidrro_el0
Let's fully exploit tpidrro_el0 by storing in it the current CPU's
struct _cpu instance alongside the userspace mode flag bit. This
greatly simplifies the code needed to get at the cpu structure, and
this paves the way to much simpler multi cluster support, as there
is no longer the need to decode MPIDR all the time.

The same code is used in the !SMP case as there are benefits there too
such as avoiding the literal pool, and it looks cleaner.

The tpidrro_el0 value is no longer stored in the exception stack frame.
Instead, we simply restore the user mode flag based on the SPSR value.
This way, more flag bits could be used independently in the future.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2021-04-14 15:06:21 -04:00
Jaxson Han
794656913b include: arm64: Add BSS align when MPU enabled
When MPU is enabled, the sections need to be 64 bytes aligned.
In the case of MMU, BSS section will be 4k aligned, because the first
variable in BSS section 'base_xlat_table' is explicitly aligned by
'__aligned(NUM_BASE_LEVEL_ENTRIES * sizeof(uint64_t))'.

However, with MPU, we do not have such a variable. So it's necessary
to fix the alignment of the BSS section in the linker.ld

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2021-04-13 07:47:44 -04:00
Jaxson Han
475374ce2f include: arm64: Add mpu data struct definition
According to Armv8-R64 Spec, MPU related meta data(region base/limit)
is 64 bits. So we need to re-define MPU related data structure here.

Signed-off-by: Haibo Xu <haibo.xu@arm.com>
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2021-04-13 07:47:44 -04:00
Henry Wang
542cd9bdbb include: arm64: Fix compile with default MMU off
If default config ARM_MMU is set to n, samples/tests will have
compilation error. This is because the arch/arm/aarch64/arm_mmu.h
is always included.

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2021-04-13 07:47:44 -04:00
Nicolas Pitre
fc8c53ff0e arm64: a few alignment fixes
The structure for the arm64_cpu_init array has to carry the cache
alignment on the whole structure and not on some internal padding
to achieve the desired effect.

And align struct __esf to a 16-byte boundary which will also align
its size accordingly. This structure is allocated on the stack on
exception entry and the ABI prescribed 16-byte stack alignment
should be preserved.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2021-04-12 11:47:41 -04:00
Krzysztof Chruscinski
8bee027ec4 arch: arm: Unconditionally compile IRQ_ZERO_LATENCY flag
Flag was present only when ZLI was enabled. That resulted in additional
ifdefs needed whenever code supports ZLI and non-ZLI mode.

Removed ifdefs, added build assert to irq connections to fail at
compile time if IRQ_ZERO_LATENCY is set but ZLI is disabled. Additional
clean up made which resulted from removing the ifdef.

Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
2021-04-12 07:33:27 -04:00
Flavio Ceolin
95cd021cea arch: arm: Fix 14.4 guideline violation
The controlling expression of an if statement has to be an
essentially boolean type.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-04-06 10:25:24 -04:00
Anas Nashif
0ec3774bde userspace: rename _is_user_context -> k_is_user_context
This functions is being called across the tree, no reason why it should
not be a public API.

The current usage violates a few MISRA rules.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-04-01 05:34:17 -04:00
Carlo Caione
a43f3bade8 arm/arm64: Fix misc and trivials for ARM/ARM64 split
Fix the header guards, comments, github labeler, CODEOWNERS and
MAINTAINERS files.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2021-03-31 10:34:33 -05:00
Carlo Caione
3539c2fbb3 arm/arm64: Make ARM64 a standalone architecture
Split ARM and ARM64 architectures.

Details:

- CONFIG_ARM64 is decoupled from CONFIG_ARM (not a subset anymore)
- Arch and include AArch64 files are in a dedicated directory
  (arch/arm64 and include/arch/arm64)
- AArch64 boards and SoC are moved to soc/arm64 and boards/arm64
- AArch64-specific DTS files are moved to dts/arm64
- The A72 support for the bcm_vk/viper board is moved in the
  boards/bcm_vk/viper directory

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2021-03-31 10:34:33 -05:00
Carlo Caione
d4066dae83 aarch64: Rework linker script for better MMU support
In acda9bf9ce new macros were introduced to support a virtual memory
map. Adjust these macros also for AArch64. This also fixes a problem
when building Zephyr when CONFIG_KERNEL_VM_BASE is not the same as
CONFIG_SRAM_BASE_ADDRESS

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2021-03-26 08:24:50 -04:00
Martin Åberg
83f733ce59 SPARC: improve fatal log
The fatal log now contains
- Trap type in human readable representation
- Integer registers visible to the program when trap was taken
- Special register values such as PC and PSR
- Backtrace with PC and SP

If CONFIG_EXTRA_EXCEPTION_INFO is enabled, then all the above is
logged. If not, only the special registers are logged.

The format is inspired by the GRMON debug monitor and TSIM simulator.
A quick guide on how to use the values is in fatal.c.

It now looks like this:

E: tt = 0x02, illegal_instruction
E:
E:       INS        LOCALS     OUTS       GLOBALS
E:   0:  00000000   f3900fc0   40007c50   00000000
E:   1:  00000000   40004bf0   40008d30   40008c00
E:   2:  00000000   40004bf4   40008000   00000003
E:   3:  40009158   00000000   40009000   00000002
E:   4:  40008fa8   40003c00   40008fa8   00000008
E:   5:  40009000   f3400fc0   00000000   00000080
E:   6:  4000a1f8   40000050   4000a190   00000000
E:   7:  40002308   00000000   40001fb8   000000c1
E:
E: psr: f30000c7   wim: 00000008   tbr: 40000020   y: 00000000
E:  pc: 4000a1f4   npc: 4000a1f8
E:
E:       pc         sp
E:  #0   4000a1f4   4000a190
E:  #1   40002308   4000a1f8
E:  #2   40003b24   4000a258

Signed-off-by: Martin Åberg <martin.aberg@gaisler.com>
2021-03-25 17:48:23 +01:00
Martin Åberg
c2b1e8d2f5 SPARC: implement ARCH_EXCEPT()
Introduce a new software trap 15 which is generated by the
ARCH_EXCEPT() function macro.

The handler for this software trap calls z_sparc_fatal_error() and
finally z_fatal_error() with "reason" and ESF as arguments.

Signed-off-by: Martin Åberg <martin.aberg@gaisler.com>
2021-03-25 17:48:23 +01:00
Kumar Gala
520ebe4d76 arch: arm: remove compat headers
These compat headers have been moved since at least v2.4.0 release so we
can now remove them.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2021-03-25 16:40:25 +01:00
Katsuhiro Suzuki
59903e2934 kernel: arch: introduce k_float_enable()
This patch introduce new API to enable FPU of thread. This is pair of
existed k_float_disable() API. And also add empty arch_float_enable()
into each architectures that have arch_float_disable(). The arc and
riscv already implemented arch_float_enable() so I do not touch
these implementations.

Motivation: Current Zephyr implementation does not allow to use FPU
on main and other system threads like as work queue. Users need to
create an other thread with K_FP_REGS for floating point programs.
Users can use FPU more easily if they can enable FPU on running
threads.

Signed-off-by: Katsuhiro Suzuki <katsuhiro@katsuster.net>
2021-03-25 14:13:23 +01:00
Carlo Caione
807991e15f AArch64: Do not use CONFIG_GEN_PRIV_STACKS
We are setting CONFIG_GEN_PRIV_STACKS when AArch64 actually uses a
statically allocated privileged stack.

This error was not captured by the tests because we only verify whether
a read/write to a privileged stack is failing, but it can fail for a lot
of reasons including when the pointer to the privileged stack is not
initialized at all, like in this case.

With this patch we deselect CONFIG_GEN_PRIV_STACKS and we fix the
mem_protect/userspace test to correctly probe the privileged stack.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2021-03-25 07:23:19 -04:00
Eugeniy Paltsev
1b41da2630 ARC: Kconfig: rename CPU_ARCV2 option to ISA_ARCV2
* Rename CPU_ARCV2 to ISA_ARCV2. That helps to avoid conflict between
  CPU families naming and ISAs naming and aligns this options
  with other ARC OSS projects.

* Generalize ARCV2 check to ARC check where it is required.

NOTE: we add ISA_ARCV2 option in a choice list as a preparation
for ISA_ARCV3 addition.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
2021-03-25 07:23:02 -04:00
Kumar Gala
95e4b3eb2c arch: arm: Add initial support for Cortex-M55 Core
Add initial support for the Cortex-M55 Core which is an implementation
of the Armv8.1-M mainline architecture and includes support for the
M‑profile Vector Extension (MVE).

The support is based on the Cortex-M33 support that already exists in
Zephyr.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2021-03-23 13:13:32 -05:00
Daniel Leung
cdd02a99a1 cmake: rename linker_final_pass.cmd to linker.cmd
Since linker_final_pass.cmd is the actual linker script being
used to link the final Zephyr binary, rename it to linker.cmd.

This also renames LINKER_PASS2 to LINKER_ZEPHYR_FINAL simply
to clarify what it is used for, instead of ambiguous pass #n.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-03-22 13:05:33 -04:00
Watson Zeng
0da8ec70dc arch: arc: enable divide zero exception
STATUS32.DZ(bit 13) is the EV_DivZero exception enable bit, and it's
not enabled by default. we need to set it explicitly to enable divide
zero exception on early boot and each thread's setup.

The DZ bit is ignored on write and read as zero when there is no
hardware division configured. So we can simply set DZ bit even if
there is no hardware division configured.

Signed-off-by: Watson Zeng <zhiwei@synopsys.com>
2021-03-19 13:56:59 -04:00
Anas Nashif
12b53d121e clock: rename z_timer_cycle_get_32 -> sys_clock_cycle_get_32
This is another API that is being used in all timer drivers and is not
internal to the clock subsystem. Remove the leading z_ and make promote
it to a cross-subsystem API.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-03-19 11:22:17 -04:00
Carlo Caione
ec70b2bc7a aarch64: userspace: Add support for page tables swapping
Introduce the necessary routines to have the user thread stack correctly
mapped and the functions to swap page tables on context switch.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2021-03-18 19:33:59 -04:00
Carles Cufi
59a51f0e09 debug: Clean up thread awareness data sections
There's no need to duplicate the linker section for each architecture.
Instead, move the section declaration to common-rom.ld.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2021-03-17 14:43:01 -05:00
Daniel Leung
273a5e670b x86: remove usage of CONFIG_KERNEL_LINK_IN_VIRT
There is no need to use this kconfig, as the phys-to-virt
offset is enough to figure out if the kernel is linked in
virtual address space in gen_mmu.py.

For code, use Z_VM_KERNEL instead.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-03-16 15:03:44 -04:00
Andrew Boie
4c7bde4311 x86: 32-bit: set ENTRY properly
The entry point is at a physical memory location; use the
physical instead of virtual address of the start symbol.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-03-16 15:03:44 -04:00
Andrew Boie
f2e4edf51a x86: set up linker ROM region for MMU
For virtual kernels this represents where the kernel
is loaded in physical memory.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-03-16 15:03:44 -04:00
Andrew Boie
acda9bf9ce linker-tool-gcc: revise for MMU support
We need to do a few things differently if we are to support
a virtual memory map, i.e. CONFIG_MMU where CONFIG_KERNEL_VM_BASE
is not the same as CONFIG_SRAM_BASE_ADDRESS.

 - All sections must be specified with a VMA and LMA, where
   VMA is the virtual address and LMA is the physical memory
   location.
 - All sections must be specified with ALIGN_WITH_INPUT to
   keep VMAs and LMAs synchronized

To do this, the existing linker macros need some adjustment:

 - GROUP_LINK_IN undefined when CONFIG_KERNEL_VM_BASE is not
   the same as CONFIG_SRAM_BASE_ADDRESS.
 - New macro GROUP_ROM_LINK_IN for text/rodata sections
 - New macro GROUP_NOLOAD_LINK_IN for bss/noinit sections
 - Implicit ALIGN_WITH_INPUT for all sections

GROUP_FOLLOWS_AT is unused anywhere in the kernel for years
now and has been removed.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-03-16 15:03:44 -04:00
Carlo Caione
a010651c65 aarch64: mmu: Add initial support for memory domains
Introduce the basic support code for memory domains. To each domain
is associated a top page table which is a copy of the global kernel
one. When a partition is added, corresponding memory range is made
private before its mapping is adjusted.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2021-03-16 08:43:19 -04:00
Peng Fan
b4f5b9e237 aarch64: reset: initialize CNTFRQ_EL0 in the highest EL
Can only be written at the highest Exception level implemented.
For example, if EL3 is the highest implemented Exception level,
CNTFRQ_EL0 can only be written at EL3.

Also move z_arm64_el_highest_plat_init to be called when is_el_highest

Signed-off-by: Peng Fan <peng.fan@nxp.com>
2021-03-11 12:24:18 +01:00
Carlo Caione
dacd176991 aarch64: userspace: Implement syscalls
This patch adds the code managing the syscalls. The privileged stack
is setup before jumping into the real syscall.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2021-03-10 14:52:50 -05:00
Carlo Caione
6978160427 aarch64: userspace: Introduce arch_is_user_context
The arch_is_user_context() function is relying on the content of the
tpidrro_el0 register to determine whether we are in user context or not.

This register is set to '1' when in EL1 and set back to '0' when user
threads are running in userspace.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2021-03-10 14:52:50 -05:00
Carlo Caione
c559591380 aarch64: stack: Rework memory stack allocations
Introduce the necessary macros and defines to have the stack regions
correctly aligned and sized.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2021-03-10 14:52:50 -05:00
Andy Ross
64cf33952d arch/xtensa: Add non-HAL caching primitives
The Xtensa L1 cache layer has straightforward semantics accessible via
single-instructions that operate on cache lines via physical
addresses.  These are very amenable to inlining.

Unfortunately the Xtensa HAL layer requires function calls to do this,
leading to significant code waste at the calling site, an extra frame
on the stack and needless runtime instructions for situations where
the call is over a constant region that could elide the loop.  This is
made even worse because the HAL library is not built with
-ffunction-sections, so pulling in even one of these tiny cache
functions has the effect of importing a 1500-byte object file into the
link!

Add our own tiny cache layer to include/arch/xtensa/cache.h and use
that instead.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-03-08 11:14:27 -05:00
Andy Ross
4da174e1f7 arch/xtensa: Remove XTOS dependency in irq_lock()
This whole file is written to assume XEA2, so there's no value to
using an abstraction call here.  Write to the RSIL instruction
directly.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-03-08 11:14:27 -05:00
Andy Ross
820c94e5dd arch/xtensa: Inline atomics
The xtensa atomics layer was written with hand-coded assembly that had
to be called as functions.  That's needlessly slow, given that the low
level primitives are a two-instruction sequence.  Ideally the compiler
should see this as an inline to permit it to better optimize around
the needed barriers.

There was also a bug with the atomic_cas function, which had a loop
internally instead of returning the old value synchronously on a
failed swap.  That's benign right now because our existing spin lock
does nothing but retry it in a tight loop anyway, but it's incorrect
per spec and would have caused a contention hang with more elaborate
algorithms (for example a spinlock with backoff semantics).

Remove the old implementation and replace with a much smaller inline C
one based on just two assembly primitives.

This patch also contains a little bit of refactoring to address the
scheme has been split out into a separate header for each, and the
ATOMIC_OPERATIONS_CUSTOM kconfig has been renamed to
ATOMIC_OPERATIONS_ARCH to better capture what it means.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-03-08 11:14:27 -05:00
Andy Ross
eb1ef50b6b arch/xtensa: General cleanup, remove dead code
There was a bunch of dead historical cruft floating around in the
arch/xtensa tree, left over from older code versions.  It's time to do
a cleanup pass.  This is entirely refactoring and size optimization,
no behavior changes on any in-tree devices should be present.

Among the more notable changes:

+ xtensa_context.h offered an elaborate API to deal with a stack frame
  and context layout that we no longer use.

+ xtensa_rtos.h was entirely dead code

+ xtensa_timer.h was a parallel abstraction layer implementing in the
  architecture layer what we're already doing in our timer driver.

+ The architecture thread structs (_callee_saved and _thread_arch)
  aren't used by current code, and had dead fields that were removed.
  Unfortunately for standards compliance and C++ compatibility it's
  not possible to leave an empty struct here, so they have a single
  byte field.

+ xtensa_api.h was really just some interrupt management inlines used
  by irq.h, so fold that code into the outer header.

+ Remove the stale assembly offsets.  This architecture doesn't use
  that facility.

All told, more than a thousand lines have been removed.  Not bad.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-03-08 11:14:27 -05:00
Peng Fan
a2ea20dd6d arch: arm: aarch64: add SMP support
With timer/gic/cache added, we could add the SMP support.
Bringup cores

Signed-off-by: Peng Fan <peng.fan@nxp.com>
2021-03-06 07:36:37 -05:00
Peng Fan
cfc7673c28 aarch64: add arch_curr_cpu
Following other ARCHs, add arch_curr_cpu in arch_inlines.h

Signed-off-by: Peng Fan <peng.fan@nxp.com>
2021-03-06 07:36:37 -05:00
Peng Fan
251b1d39ac arch: arm: aarch64: export z_arm64_mmu_init for SMP
Export z_arm64_mmu_init for SMP usage

Signed-off-by: Peng Fan <peng.fan@nxp.com>
2021-03-06 07:36:37 -05:00
Carlo Caione
9d908c78fa aarch64: Rewrite reset code using C
There is no strict reason to use assembly for the reset routine. Move as
much code as possible to C code using the proper helpers.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2021-03-04 06:51:48 -05:00
Carlo Caione
bba7abe975 aarch64: Use helpers instead of inline assembly
No need to rely on inline assembly when helpers are available.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2021-03-04 06:51:48 -05:00
Carlo Caione
708b9b4cc9 aarch64: lib_helpers: Introduce helpers for common assembly operations
Introduce C helpers for common assembly operations and preprocessor
constructors to create helper functions to read / write system
registers.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2021-03-04 06:51:48 -05:00
Carlo Caione
a2226f5200 aarch64: Fix registers naming in cpu.h
The name for registers and bit-field in the cpu.h file is incoherent and
messy. Refactor the whole file using the proper suffixes for bits,
shifts and masks.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2021-03-04 06:51:48 -05:00
Carlo Caione
13ef690192 aarch64: linker: Remove unused symbols and sections
Remove unused symbols and related sections from the linker script. In
particular CONFIG_IS_BOOTLOADER and CONFIG_CODE_DATA_RELOCATION are not
currently supported on AArch64.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2021-03-03 11:22:25 +03:00
Nicolas Pitre
7babb12ccb aarch64: restore natural order in struct __esf
The funky order was necessary when registers were stored using a
sequence of `stp x(n), x(n+1), [sp, #-16]!`. Now that absolute offsets
are being used, we can restore natural ordering.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2021-03-02 21:04:15 +03:00
Carlo Caione
eb72b2d72a aarch64: smccc: Retrieve up to 8 64-bit values
The most common secure monitor firmware in the ARM world is TF-A. The
current release allows up to 8 64-bit values to be returned from a
SMC64 call from AArch64 state.

Extend the number of possible return values from 4 to 8.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2021-03-01 09:26:56 -05:00
Carlo Caione
bc7cb75a82 aarch64: smccc: Use offset macros
Instead of relying on hardcoded offset in the assembly code, introduce
the offset macros to make the code more clear.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2021-03-01 09:26:56 -05:00
Carlo Caione
6ff37886dd aarch64: mem_domain: Introduce k_mem_partition_attr_t
Introduce the kernel macros for memory attribution also for AArch64.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2021-03-01 09:25:44 -05:00
Nicolas Pitre
cb49e4b789 aarch64: mmu: invert the MT_OVERWRITE flag
The MT_OVERWRITE case is much more common. Redefine that flag as
MT_NO_OVERWRITE instead for those fewer cases where it is needed.

One such case is platform provided mappings. Apply them after the
common kernel mappings and use the MT_NO_OVERWRITE on them.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2021-02-28 16:49:12 -05:00
Nicolas Pitre
b696090bb7 aarch64: mmu: make page table pool global
There is no real reason for keeping page tables into separate pools.
Make it global which allows for more efficient memory usage and
simplifies the code.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2021-02-28 16:49:12 -05:00
Nicolas Pitre
459bfed9ea aarch64: mmu: dynamic mapping support
Introduce a remove_map() to ... remove a mapping.

Add a use count to the page table pool so pages can be dynamically
allocated, deallocated and reused.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2021-02-28 16:49:12 -05:00
Antony Pavlov
f463bcaa9d include: arch: riscv: drop __soc_get_irq() declaration
There is no __soc_get_irq() definition in the Zephyr source
code tree.

Signed-off-by: Antony Pavlov <antonynpavlov@gmail.com>
2021-02-25 21:52:20 +03:00
Shih-Wei Teng
a418b2535b RISC-V: Enhancement for stack size to 16-bytes aligned at least
If enable CONFIG_PMP_STACK_GUARD, the stack size may not 16-bytes
aligned. Enhance this issue.

Signed-off-by: Shih-Wei Teng <swteng@andestech.com>
2021-02-24 10:37:03 -05:00
Yuguo Zou
a8b6936c7d arch: arc: fix mpu version number
ARC mpu version used a wrong number 3, could cause conflict in future.
This commit fix this issue to version number 4.

Signed-off-by: Yuguo Zou <yuguo.zou@synopsys.com>
2021-02-24 08:57:35 -05:00
Daniel Leung
2816c17a09 x86: allow linking in virtual address space
This adds the pieces to allow the kernel to be linked
in virtual address space.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-02-22 14:55:28 -05:00
Daniel Leung
d340afd456 x86: use CONFIG_SRAM_OFFSET instead of CONFIG_X86_KERNEL_OFFSET
This changes x86 to use CONFIG_SRAM_OFFSET instead of
arch-specific CONFIG_X86_KERNEL_OFFSET. This allows the common
MMU macro Z_BOOT_VIRT_TO_PHYS() and Z_BOOT_PHYS_TO_VIRT() to
function properly if we ever need to map the kernel into
virtual address space that does not have the same starting
physical address.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-02-22 14:55:28 -05:00