d554d34137
Following the idiom used for system calls, add script support to read the initial application binary to identify which devices are defined, and to use their offset in the device array as their unique handle rather than the externally-defined ordinal from devicetree. The device dependency arrays are updated to use these handles. Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
291 lines
9.3 KiB
ReStructuredText
291 lines
9.3 KiB
ReStructuredText
.. _build_overview:
|
||
|
||
Build Overview
|
||
##############
|
||
|
||
The Zephyr build process can be divided into two main phases: a configuration
|
||
phase (driven by CMake) and a build phase (driven by Make or Ninja).
|
||
|
||
.. _build_configuration_phase:
|
||
|
||
Configuration Phase
|
||
*******************
|
||
|
||
The configuration phase begins when the user invokes *CMake*,
|
||
specifying a source application directory and a board target.
|
||
|
||
.. figure:: build-config-phase.svg
|
||
:align: center
|
||
:alt: Zephyr's build configuration phase
|
||
:figclass: align-center
|
||
:width: 80%
|
||
|
||
CMake begins by processing the :file:`CMakeLists.txt` file in the application
|
||
directory, which refers to the :file:`CMakeLists.txt` file in the Zephyr
|
||
top-level directory, which in turn refers to :file:`CMakeLists.txt` files
|
||
throughout the build tree (directly and indirectly). Its primary output is a
|
||
set of Makefiles or Ninja files to drive the build process, but the CMake
|
||
scripts also do some processing of their own:
|
||
|
||
Devicetree
|
||
:file:`*.dts` (*devicetree source*) and :file:`*.dtsi` (*devicetree source
|
||
include*) files are collected from the target's architecture, SoC, board,
|
||
and application directories.
|
||
|
||
:file:`*.dtsi` files are included by :file:`*.dts` files via the C
|
||
preprocessor (often abbreviated *cpp*, which should not be confused with
|
||
C++). The C preprocessor is also used to merge in any devicetree
|
||
:file:`*.overlay` files, and to expand macros in :file:`*.dts`,
|
||
:file:`*.dtsi`, and :file:`*.overlay` files.
|
||
|
||
The preprocessed devicetree sources (stored in :file:`*.dts.pre.tmp`) are
|
||
parsed by :zephyr_file:`gen_defines.py <scripts/dts/gen_defines.py>` to
|
||
generate a :file:`devicetree_unfixed.h` header with preprocessor macros.
|
||
|
||
As a debugging aid, :file:`gen_defines.py` writes the final devicetree to
|
||
:file:`zephyr.dts`. This file is just for reference. It is not used
|
||
anywhere.
|
||
|
||
The ``dtc`` devicetree compiler also gets run on the preprocessed devicetree
|
||
sources to catch any extra warnings and errors generated by it. The output
|
||
from ``dtc`` is unused otherwise.
|
||
|
||
The above is just a brief overview. For more information on devicetree, see
|
||
:ref:`dt-guide`.
|
||
|
||
Devicetree fixups
|
||
Files named :file:`dts_fixup.h` from the target’s architecture, SoC, board,
|
||
and application directories are concatenated into a single
|
||
:file:`devicetree_fixups.h` file. :file:`dts_fixup.h` files are used to
|
||
rename generated macros to names expected by the source code.
|
||
|
||
Source code accesses preprocessor macros generated from devicetree by
|
||
including the :zephyr_file:`devicetree.h <include/devicetree.h>` header,
|
||
which includes :file:`devicetree_unfixed.h` and :file:`devicetree_fixups.h`.
|
||
|
||
Kconfig
|
||
:file:`Kconfig` files define available configuration options for for the
|
||
target architecture, SoC, board, and application, as well as dependencies
|
||
between options.
|
||
|
||
Kconfig configurations are stored in *configuration files*. The initial
|
||
configuration is generated by merging configuration fragments from the board
|
||
and application (e.g. :file:`prj.conf`).
|
||
|
||
The output from Kconfig is an :file:`autoconf.h` header with preprocessor
|
||
assignments, and a :file:`.config` file that acts both as a saved
|
||
configuration and as configuration output (used by CMake).
|
||
|
||
Information from devicetree is available to Kconfig, through the functions
|
||
defined in :zephyr_file:`kconfigfunctions.py
|
||
<scripts/kconfig/kconfigfunctions.py>`.
|
||
|
||
See :ref:`the Kconfig section of the manual <kconfig>` for more information.
|
||
|
||
Build Phase
|
||
***********
|
||
|
||
The build phase begins when the user invokes ``make`` or ``ninja``. Its
|
||
ultimate output is a complete Zephyr application in a format suitable for
|
||
loading/flashing on the desired target board (:file:`zephyr.elf`,
|
||
:file:`zephyr.hex`, etc.) The build phase can be broken down, conceptually,
|
||
into four stages: the pre-build, first-pass binary, final binary, and
|
||
post-processing.
|
||
|
||
Pre-build
|
||
=========
|
||
|
||
Pre-build occurs before any source files are compiled, because during
|
||
this phase header files used by the source files are generated.
|
||
|
||
Offset generation
|
||
Access to high-level data structures and members is sometimes
|
||
required when the definitions of those structures is not
|
||
immediately accessible (e.g., assembly language). The generation of
|
||
*offsets.h* (by *gen_offset_header.py*) facilitates this.
|
||
|
||
System call boilerplate
|
||
The *gen_syscall.py* and *parse_syscalls.py* scripts work
|
||
together to bind potential system call functions with their
|
||
implementations.
|
||
|
||
.. figure:: build-build-phase-1.svg
|
||
:align: center
|
||
:alt: Zephyr's build stage I
|
||
:figclass: align-center
|
||
:width: 80%
|
||
|
||
First-pass binary
|
||
=================
|
||
|
||
Compilation proper begins with the first-pass binary. Source files (C
|
||
and assembly) are collected from various subsystems (which ones is
|
||
decided during the configuration phase), and compiled into archives
|
||
(with reference to header files in the tree, as well as those
|
||
generated during the configuration phase and the pre-build stage).
|
||
|
||
If memory protection is enabled, then:
|
||
|
||
Partition grouping
|
||
The *gen_app_partitions.py* script scans all the
|
||
generated archives and outputs linker scripts to ensure that
|
||
application partitions are properly grouped and aligned for the
|
||
target’s memory protection hardware.
|
||
|
||
Then *cpp* is used to combine linker script fragments from the target’s
|
||
architecture/SoC, the kernel tree, optionally the partition output if
|
||
memory protection is enabled, and any other fragments selected during
|
||
the configuration process, into a *linker.cmd* file. The compiled
|
||
archives are then linked with *ld* as specified in the
|
||
*linker.cmd*.
|
||
|
||
In some configurations, this is the final binary, and the next stage
|
||
is skipped.
|
||
|
||
.. figure:: build-build-phase-2.svg
|
||
:align: center
|
||
:alt: Zephyr's build stage II
|
||
:figclass: align-center
|
||
:width: 80%
|
||
|
||
Final binary
|
||
============
|
||
|
||
The binary from the previous stage is incomplete, with empty and/or
|
||
placeholder sections that must be filled in by, essentially, reflection.
|
||
|
||
Device dependencies
|
||
The *gen_handles.py* script scans the first-pass binary to determine
|
||
relationships between devices that were recorded from devicetree data,
|
||
and replaces the encoded relationships with values that are optimized to
|
||
locate the devices actually present in the application.
|
||
|
||
When :ref:`usermode_api` is enabled:
|
||
|
||
Kernel object hashing
|
||
The *gen_kobject_list.py* scans the *ELF DWARF*
|
||
debug data to find the address of the all kernel objects. This
|
||
list is passed to *gperf*, which generates a perfect hash function and
|
||
table of those addresses, then that output is optimized by
|
||
*process_gperf.py*, using known properties of our special case.
|
||
|
||
Then, the link from the previous stage is repeated, this time with the
|
||
missing pieces populated.
|
||
|
||
.. figure:: build-build-phase-3.svg
|
||
:align: center
|
||
:alt: Zephyr's build stage III
|
||
:figclass: align-center
|
||
:width: 80%
|
||
|
||
|
||
Post processing
|
||
===============
|
||
|
||
Finally, if necessary, the completed kernel is converted from *ELF* to
|
||
the format expected by the loader and/or flash tool required by the
|
||
target. This is accomplished in a straightforward manner with *objdump*.
|
||
|
||
.. figure:: build-build-phase-4.svg
|
||
:align: center
|
||
:alt: Zephyr's build final stage
|
||
:figclass: align-center
|
||
:width: 80%
|
||
|
||
|
||
.. _build_system_scripts:
|
||
|
||
Supporting Scripts and Tools
|
||
****************************
|
||
|
||
The following is a detailed description of the scripts used during the build process.
|
||
|
||
.. _gen_syscalls.py:
|
||
|
||
:zephyr_file:`scripts/gen_syscalls.py`
|
||
========================================
|
||
|
||
.. include:: ../../../scripts/gen_syscalls.py
|
||
:start-after: """
|
||
:end-before: """
|
||
|
||
.. _gen_handles.py:
|
||
|
||
:zephyr_file:`scripts/gen_handles.py`
|
||
==========================================
|
||
|
||
.. include:: ../../../scripts/gen_handles.py
|
||
:start-after: """
|
||
:end-before: """
|
||
|
||
.. _gen_kobject_list.py:
|
||
|
||
:zephyr_file:`scripts/gen_kobject_list.py`
|
||
==========================================
|
||
|
||
.. include:: ../../../scripts/gen_kobject_list.py
|
||
:start-after: """
|
||
:end-before: """
|
||
|
||
.. _gen_offset_header.py:
|
||
|
||
:zephyr_file:`scripts/gen_offset_header.py`
|
||
===========================================
|
||
|
||
.. include:: ../../../scripts/gen_offset_header.py
|
||
:start-after: """
|
||
:end-before: """
|
||
|
||
.. _parse_syscalls.py:
|
||
|
||
:zephyr_file:`scripts/parse_syscalls.py`
|
||
========================================
|
||
|
||
|
||
.. include:: ../../../scripts/parse_syscalls.py
|
||
:start-after: """
|
||
:end-before: """
|
||
|
||
.. _gen_idt.py:
|
||
|
||
:zephyr_file:`arch/x86/gen_idt.py`
|
||
==================================
|
||
|
||
.. include:: ../../../arch/x86/gen_idt.py
|
||
:start-after: """
|
||
:end-before: """
|
||
|
||
.. _gen_gdt.py:
|
||
|
||
:zephyr_file:`arch/x86/gen_gdt.py`
|
||
===================================
|
||
|
||
.. include:: ../../../arch/x86/gen_gdt.py
|
||
:start-after: """
|
||
:end-before: """
|
||
|
||
.. _gen_relocate_app.py:
|
||
|
||
:zephyr_file:`scripts/gen_relocate_app.py`
|
||
===========================================
|
||
|
||
.. include:: ../../../scripts/gen_relocate_app.py
|
||
:start-after: """
|
||
:end-before: """
|
||
|
||
.. _process_gperf.py:
|
||
|
||
:zephyr_file:`scripts/process_gperf.py`
|
||
========================================
|
||
|
||
.. include:: ../../../scripts/process_gperf.py
|
||
:start-after: """
|
||
:end-before: """
|
||
|
||
:zephyr_file:`scripts/gen_app_partitions.py`
|
||
============================================
|
||
|
||
.. include:: ../../../scripts/gen_app_partitions.py
|
||
:start-after: """
|
||
:end-before: """
|