For those still using old variable ZEPHYR_GCC_VARIANT.
raise an error if the variable is not defined.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add the STM32 ccm_bss, ccm_noinit, abd ccm_data sections
to the list of allowed sections so the sanity check passes.
Signed-off-by: Erwin Rol <erwin@erwinrol.com>
We want to support other toolchain not based on GCC, so the variable is
confusing, use ZEPHYR_TOOLCHAIN_VARIANT instead.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
It is important that it is easy to reproduce CI issues locally. Using
the same sanitycheck options locally and in CI helps in this
regard. Specifically, Ninja and Make can produce different results and
therefore the default generator should be the same for sanitycheck and
shippable.
This patch makes four changes:
The sanitycheck option '--make' is introduced to allow specifying Make
as a generator.
CI no longer passes the option '--ninja' to sanitycheck.
Sanitycheck defaults to using Ninja.
Sanitycheck documents the --ninja option as deprecated.
Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
When generating coverage report:
* Also generate branch coverage
* If the report is properly generated tell user where to find it
* Handle case in which no ztests are run (it would crash before)
Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
Aligned sanitycheck command line options in doc
with the actual ones.
+
Minor fix in --coverage description.
Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
sanitycheck: Compile unit tests with coverage enabled always
+ run also first unit tests together with native_posix
shippable: also include unit_testing coverage into report to
codecov
Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
This patch adds app_pad and priv_stacks to the rw section list so that
tests which validate the sections found in the binary pass when run on
systems which contain MPUs.
Signed-off-by: Andy Gross <andy.gross@linaro.org>
Rather than having handler generate differently named log files,
generate the same for all handlers, so qemu, native and unit will now
generate handler.log.
This will simplify things and reporting will be consistent. Also, this
fixes a bug where we only included qemu.log in generated reports.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When building the testcase path, replace symlinks in path
with the realpath both in the testcase root and in ZEPHYR_BASE
For more info see issue #5772Fixes: #5772
Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
Changed the default behavior for when we run sanitycheck to match the -R
flag that turns on assertions. Introduced a --disable-asserts option if
we desire to explicitly turn of asserts.
This matches behavior that our CI builds have been doing and addresses
part of #5726.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Add 2 classes, one to handle the current TestCase scenario, and one more
for handling generic Console with regex matching.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Some boards are supported natively by qemu. This option will allow us to
run tests using those platforms directly without having to go via a
dedicated qemu board definition.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
if we are using command line platform filter, no need to list every
other platform as excluded, we know that already. Show only the
discards that apply to the selected platforms on the command line
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When we load tests from a file, we do not have the discarded list, this
would have been done already when saving the test file.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When checking if a testcase passed or failed, allow there
to be prefixes or postfixes in the line, around
PROJECT EXECUTION SUCCESSFUL/FAILED
Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
CONFIG_ASSERT is being set by cmake, so it is not possible to filter
using the generated config, add this as a standalone filter.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Generating coverage data is split over two CI jobs which means the
service will need to merge results and reports wrong coverage data when
only 1 job is finished. This puts the native_posix board first making
sure we run on the first job and generate data in one place.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Extend exception handling to cover not just YAML loading, but any
error while accessing parsed data too. That may catch e.g. schema
mismatch errors (for folks who don't have pykwalify installed, which
is optional). So, now error will be logged, but processing of other
tests will continue.
For example, I had a local, uncommitted test which wasn't converted
per 23f81eeb42 and caused:
Traceback (most recent call last):
File "./scripts/sanitycheck", line 2456, in <module>
main()
File "./scripts/sanitycheck", line 2324, in main
options.outdir, options.coverage)
File "./scripts/sanitycheck", line 1445, in __init__
for name in parsed_data.tests.keys():
AttributeError: 'list' object has no attribute 'keys'
Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
We have been passing around options from one function to the next making
it very difficult to add a new option easily and requiring changes to
man function prototypes.
This declated the parsed command line options global and renames args to
options. args is being used elsewhere and this was confusing.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This keyword would mean that a special harness is needed to run the
tests sucessfully. This can be as simple as a loopback wiring or a
complete hardware test setup for sensor and IO testing. It is free form
initially and would be changed to be an enum once we have more values in
place.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
We have now different runners/handlers, so avoid using qemu terminology
for the generic classes and for generic usage.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Passing ARCH during the build process is something from the past and
samples/tests should not do that, remove it here to catch any
violations.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Since the move to YAML format and the change in how we define default
platforms this is no longer needed as we are able to set multiple
default platforms per architecture and not using a list based on
priority anymore.
Fixes#4445
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The section terminology was relevant with the ini syntax, with yaml we
can call this a test and avoid confusion and make the code more
readable.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Simplify parsing of yaml structures and remove usage of cp which was for
the ConfigParser used for ini files.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
It is supported to add give extra flags to the linker from the
commandline like this:
cmake -DEXTRA_LDFLAGS=-Lmy_dir path
But unfortunately this was broken during the CMake
migration. Interestingly, the reason that it was broken is that KBuild
was also partially broken. KBuild would pass on EXTRA_LDFLAGS when
object files were linked together into built-in.o files, but it would
not use EXTRA_LDFLAGS for the final link into an elf file.
This patch fixes EXTRA_LDFLAGS.
Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
This can be used by other handlers and is defined in the main Handler
class. Qemu is just an implementer.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This makes piped output work as the user expects. And looking at the
piped output is the only way to use sanitycheck normally because
of #4603.
Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
sanitycheck was incorrectly documenting that --extra-args would pass
on it's input unchanged to Make.
In reality --extra-args acts as a way to define extra CMake cache
entries. The key-value entries will be prefixed with -D before being
passed to CMake.
E.g
"sanitycheck -x=USE_CCACHE=0"
will translate to
"cmake -DUSE_CCACHE=0"
Signed-off-by: Sebastian Boe <sebastian.boe@nordicsemi.no>
