Commit Graph

142 Commits

Author SHA1 Message Date
Anas Nashif
4d25b50c9c sanitycheck: remove usage of qemu for generic handlers
We have now different runners/handlers, so avoid using qemu terminology
for the generic classes and for generic usage.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-12-11 14:47:08 -05:00
Anas Nashif
f487699810 sanitycheck: descriptive variable names
Use qemu instead of q and make code more readable.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-12-11 14:47:08 -05:00
Anas Nashif
305a88eb00 sanitycheck: we do not need to pass ARCH= on command line
Passing ARCH during the build process is something from the past and
samples/tests should not do that, remove it here to catch any
violations.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-12-11 14:47:08 -05:00
Anas Nashif
ab74706e2c sanitycheck: remove platform-level option
Since the move to YAML format and the change in how we define default
platforms this is no longer needed as we are able to set multiple
default platforms per architecture and not using a list based on
priority anymore.

Fixes #4445

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-12-11 14:47:08 -05:00
Anas Nashif
877d3caee3 sanitycheck: reworked Platform/Architecture classes
Fix documentation of both classes and minor cleanup.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-12-11 14:47:08 -05:00
Anas Nashif
b4754ede52 sanitycheck: use test instead of section
The section terminology was relevant with the ini syntax, with yaml we
can call this a test and avoid confusion and make the code more
readable.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-12-11 14:47:08 -05:00
Anas Nashif
3ba1d43847 sanitycheck: flake8 fixes and move closer to pep8
run through flake8 and autopep8 scripts and cleanup as much as possible.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-12-11 14:47:08 -05:00
Anas Nashif
255625b0a3 sanitycheck: refactor data parsing in script
Simplify parsing of yaml structures and remove usage of cp which was for
the ConfigParser used for ini files.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-12-11 14:47:08 -05:00
Sebastian Bøe
abbdd88683 cmake: Fix EXTRA_LDFLAGS feature
It is supported to add give extra flags to the linker from the
commandline like this:

cmake -DEXTRA_LDFLAGS=-Lmy_dir path

But unfortunately this was broken during the CMake
migration. Interestingly, the reason that it was broken is that KBuild
was also partially broken. KBuild would pass on EXTRA_LDFLAGS when
object files were linked together into built-in.o files, but it would
not use EXTRA_LDFLAGS for the final link into an elf file.

This patch fixes EXTRA_LDFLAGS.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2017-12-05 08:39:43 -05:00
Anas Nashif
5fd499b3c6 sanitycheck: remove obsolete DISABLE_TRYRUN
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-11-28 15:31:27 -05:00
Anas Nashif
66bdb3223e sanitycheck: fix documentation and remove mention of kbuild
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-11-28 15:31:27 -05:00
Anas Nashif
c8390f1b49 sanitycheck: rename qemu_time to handler time
This can be used by other handlers and is defined in the main Handler
class. Qemu is just an implementer.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-11-28 15:31:27 -05:00
Anas Nashif
8b11a1f207 sanitycheck: use glob to find board yaml file
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-11-28 15:31:27 -05:00
Paul Sokolovsky
3ba0876a3f sanitycheck: Flush stdout in info()
This makes piped output work as the user expects. And looking at the
piped output is the only way to use sanitycheck normally because
of #4603.

Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
2017-11-13 19:31:19 -05:00
Sebastian Bøe
c218261a33 sanitycheck: Fixed documentation for --extra-args and extra_args
sanitycheck was incorrectly documenting that --extra-args would pass
on it's input unchanged to Make.

In reality --extra-args acts as a way to define extra CMake cache
entries. The key-value entries will be prefixed with -D before being
passed to CMake.

E.g
"sanitycheck -x=USE_CCACHE=0"
will translate to
"cmake -DUSE_CCACHE=0"

Signed-off-by: Sebastian Boe <sebastian.boe@nordicsemi.no>
2017-11-09 17:55:30 -05:00
Sebastian Bøe
71d7de01ef sanitycheck: Minor internal documentation fixes
Signed-off-by: Sebastian Boe <sebastian.boe@nordicsemi.no>
2017-11-09 17:55:30 -05:00
Sebastian Bøe
781e39887c sanitycheck: Remove the --ccache option from sanitycheck
The --ccache option will be replaced by another mechanism.

Signed-off-by: Sebastian Boe <sebastian.boe@nordicsemi.no>
2017-11-09 17:55:30 -05:00
Anas Nashif
fb91ad6b88 sanitycheck: adapt for cmake
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-11-08 20:00:22 -05:00
Andrew Boie
8c20db7d44 sanitycheck: tone down verbose output
We only had a few hundred tests run when sanitycheck was first written,
and printing out the reasoning why tests were skipped seemed reasonable
at the time. Now that we are running tens of thousands of tests, this
is too much information.

The dump of what tests were skipped and why now requires two instances
of --verbose on the command line.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2017-10-27 10:55:12 -07:00
Andrew Boie
877f82e847 userspace: add K_THREAD_ACCCESS_GRANT()
It's possible to declare static threads that start up as K_USER,
but these threads can't do much since they start with permissions on
no kernel objects other than their own thread object.

Rather than do some run-time synchronization to have some other thread
grant the necessary permissions, we introduce macros
to conveniently assign object permissions to these threads when they
are brought up at boot by the kernel. The tables generated here
are constant and live in ROM when possible.

Example usage:

K_THREAD_DEFINE(my_thread, STACK_SIZE, my_thread_entry,
                NULL, NULL, NULL, 0, K_USER, K_NO_WAIT);

K_THREAD_ACCESS_GRANT(my_thread, &my_sem, &my_mutex, &my_pipe);

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2017-10-18 07:37:38 -07:00
Anas Nashif
981f77f756 sanitycheck: fix extra_configs with multiple values
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-10-18 08:33:03 -04:00
Anas Nashif
ebc329dc30 doc: document extra_configs option for testcases
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-10-17 14:13:52 -04:00
Anas Nashif
fa695d2b9e sanitycheck: add extra_configs for testing with multiple values
Support new keywords in testcase.yaml that would allow us to inject
configuration options to be merged with default configuration instead of
having to provide a prj.conf for each variant of the test which is very
difficult to keep in sync.  Sanitycheck script will create an overlay
file that is merged during the build process.

This is now done using the extra_configs option which is a yaml list of
option with the values, for example:

 extra_configs:
   - CONFIG_XXXX=y
   - CONFIG_YYYY=y

With this option we can have multiple tests that for example run on
hardware with different values. This type of testing is good on HW but
it does not make sense to be built in normal sanitycheck operation
because it will be just rebuilding the same code with different values.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-10-17 08:55:38 -04:00
Anas Nashif
86c8e2330d sanitycheck: fix support for unit tests
When we moved to yaml, we missed porting the 'unit' architecture. This
is now being added again.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-10-11 09:21:05 -05:00
Anas Nashif
7fae29c711 sanitycheck: inifile -> yamlfile
because we do not use ini files anymore, to avoid confusion, rename this
to be yamlfile, which is the format we use for testcases now.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-10-11 09:21:05 -05:00
Anas Nashif
bd166f4903 sanitycheck: save/load lists of filterd tests
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-09-12 18:13:46 -04:00
Anas Nashif
70783cab60 sanitycheck: rename arch_root -> board_root
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-09-12 18:13:46 -04:00
Anas Nashif
db3d55ff1b sanitycheck: move all related files to sanity_chk
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-09-11 08:53:56 -07:00
Andrew Boie
945af95f42 kernel: introduce object validation mechanism
All system calls made from userspace which involve pointers to kernel
objects (including device drivers) will need to have those pointers
validated; userspace should never be able to crash the kernel by passing
it garbage.

The actual validation with _k_object_validate() will be in the system
call receiver code, which doesn't exist yet.

- CONFIG_USERSPACE introduced. We are somewhat far away from having an
  end-to-end implementation, but at least need a Kconfig symbol to
  guard the incoming code with. Formal documentation doesn't exist yet
  either, but will appear later down the road once the implementation is
  mostly finalized.

- In the memory region for RAM, the data section has been moved last,
  past bss and noinit. This ensures that inserting generated tables
  with addresses of kernel objects does not change the addresses of
  those objects (which would make the table invalid)

- The DWARF debug information in the generated ELF binary is parsed to
  fetch the locations of all kernel objects and pass this to gperf to
  create a perfect hash table of their memory addresses.

- The generated gperf code doesn't know that we are exclusively working
  with memory addresses and uses memory inefficently. A post-processing
  script process_gperf.py adjusts the generated code before it is
  compiled to work with pointer values directly and not strings
  containing them.

- _k_object_init() calls inserted into the init functions for the set of
  kernel object types we are going to support so far

Issue: ZEP-2187
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2017-09-07 16:33:33 -07:00
Andrew Boie
3ef0b56c15 sanitycheck: fix rw_sections whitespace
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2017-09-07 16:33:33 -07:00
Anas Nashif
6222418337 sanitycheck: fix RAM check
make this consistent with flash size check. This issue caused platforms
with 8k to be completelty ignored.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-08-14 13:28:42 -04:00
Inaky Perez-Gonzalez
662dde6675 sanitycheck: validate YAML files w/ schemas files
This introduces an schema-based YAML validation process when loading
any YAML file, before doing any operations on them. An exception will
be raised at SanityConfigParser() if the file fails to verify with the
given schema.

Schemas are defined for the platform files in board///*.yaml and for
the (sample|testcase).yaml files. The verification is done using the
pykwalify python library. If not installed, a warning is printed and
the verification schema is skipped. At some point, we might want to
force it being installed.

The verification library is made a separate module (scl.py) so it can
be easily imported by others.

Signed-off-by: Inaky Perez-Gonzalez <inaky.perez-gonzalez@intel.com>
2017-08-07 21:19:12 -04:00
Sebastian Bøe
d3409c5e67 doc: scripts: sanitycheck: s/ini/yaml/
AFAIK an ini file system was ported to a yaml file system. But some
ini file references still remain.

This patch changes all ini file mentions into yaml.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2017-08-07 12:58:49 -04:00
Anas Nashif
3ac7b3a229 doc: qemu target was deprecated, use 'run'
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-08-03 11:48:55 -04:00
Anas Nashif
9527693f2a sanitycheck: support feature classes for boards
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-08-02 07:31:22 -04:00
Andrew Boie
65a9d2a94a kernel: make K_.*_INITIALIZER private to kernel
Upcoming memory protection features will be placing some additional
constraints on kernel objects:

- They need to reside in memory owned by the kernel and not the
application
- Certain kernel object validation schemes will require some run-time
initialization of all kernel objects before they can be used.

Per Ben these initializer macros were never intended to be public. It is
not forbidden to use them, but doing so requires care: the memory being
initialized must reside in kernel space, and extra runtime
initialization steps may need to be peformed before they are fully
usable as kernel objects. In particular, kernel subsystems or drivers
whose objects are already in kernel memory may still need to use these
macros if they define kernel objects as members of a larger data
structure.

It is intended that application developers instead use the
K_<object>_DEFINE macros, which will automatically put the object in the
right memory and add them to a section which can be iterated over at
boot to complete initiailization.

There was no K_WORK_DEFINE() macro for creating struct k_work objects,
this is now added.

k_poll_event and k_poll_signal are intended to be instatiated from
application memory and have not been changed.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2017-07-10 11:44:56 -07:00
Kumar Gala
5141d526b3 sanitycheck: fix depends_on when multiple dependencies
If the depends_on has more than one item we need to match all of those
dependencies in the supported list.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2017-07-07 09:01:01 -05:00
Anas Nashif
25a8b88588 sanitycheck: fix supported/depends_on usage
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-07-05 12:43:13 -04:00
Andrew Boie
8eed4b0309 sections.h: define application-level sections
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2017-06-29 07:46:58 -04:00
Anas Nashif
b17e1caf0a sanitycheck: add toolchain keyword to testcases
Some testcases can only be built with certain toolchains. Instead of
using filters, add support for toolchain keyword which enables
whitelisting and exclusion.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-06-28 09:51:40 -05:00
Kumar Gala
914d92e488 scripts/sanitycheck: split out log from config-sanitycheck
Put the results of the config-sanitycheck into their own log so we can
see warnings from that stage of the build.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2017-06-27 19:11:31 -04:00
Anas Nashif
bfab06bb52 sanitycheck: fix platform filtering
The build_on_all tag in synchronisation sample was resetting the
supplied arguemnt for filtering platforms.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-06-22 09:48:48 -04:00
Anas Nashif
cf21f5f10b sanitycheck: enhance toolchain filtering
Do not run filters on platforms not supported by toolchain.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-06-22 09:22:48 -04:00
Anas Nashif
a792a3d410 sanitycheck: support testcases in yaml instead of ini
This commit changes the syntax of the testcase files and changes the
behaviour and configuration of the sanitycheck script.

To avoid having multiple files with different syntax for boards,
samples, tests; this change unifies the syntax and uses YAML instead of
INI.

We maintain the current keywords used in the old syntax and maintain the
flexibility of adding tests with different configuration by using YAML
list configuration. On top of that, the following features are added:

- We now scan for board configurations in the boards directory and look
for a YAML file describing a board and how it should be tested. This
eliminates the need for listing boards per architecture in a special ini
file under scripts/.

- We define hardware information charachterstics in the board YAML file
that helps identifying if a certain test should run on that board or
not. For example, we can specify the available RAM in the board and
filter tests that would require more RAM than the board can handle.

- Boards can be set as default for testing meaning that we always run a
test case (build and run of possible) when sanitycheck is called without
any arguments. Previously this was done only by selecting the first
board defined for a specific architecture.

- Tests can be configured to run on all possible boards, this is to make
sure we always build some basic tests for all boards to catch issues
with the core kernel features.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-06-21 20:56:53 -04:00
David B. Kinder
29963c30d1 scripts: sanitycheck: more spelliing fixes
PR #515 got merged before I could submit some additional spelling
fixes.

Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
2017-06-16 17:15:57 -04:00
Johan Hedberg
9703927f84 net: buf: Move net_buf_pool objects to dedicated linker area
Moving the net_buf_pool objects to a dedicated area lets us access
them by array offset into this area instead of directly by pointer.
This helps reduce the size of net_buf objects by 4 bytes.

Signed-off-by: Johan Hedberg <johan.hedberg@intel.com>
2017-06-16 17:05:06 +03:00
Paul Sokolovsky
ff70add7ab scripts: sanitycheck: Clean up help text.
Fixes a typo, punctuation, and a bit of old info about test paths.

Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
2017-06-15 23:04:12 -04:00
Andy Gross
25309ab51a scripts: sanitycheck: Switch config target
This patch changes the config target to use the config-sanitycheck
target.  The config-sanitycheck target gets not only the Kconfig
options, but also the DTS generated options.

Signed-off-by: Andy Gross <andy.gross@linaro.org>
2017-06-15 16:14:59 -05:00
Anas Nashif
e6fcc0105e sanitycheck: remove color output from xml
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-05-17 09:10:42 -05:00
Anas Nashif
0605fa33ea sanitycheck: junit: add total duration of run
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-05-15 15:54:50 -04:00