Commit Graph

269 Commits

Author SHA1 Message Date
Flavio Ceolin
e6fa658a57 tests: mbedtls: Follow changes in mbedTLS 3.0 version
Apply the modifications required by the newer library version.
These modifications came from the original file in the mbedTLS
repository.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-10-07 14:02:40 -05:00
Chen Peng1
47670d1ba6 tests: benchmarks: latency_measure: fix potential div by zero
If the count is zero in the heap_malloc_free test, a div by zero
would happen, so add a condition to handle this potential error.

Signed-off-by: Chen Peng1 <peng1.chen@intel.com>
2021-09-08 10:28:58 -04:00
Stephanos Ioannidis
96c7f6ab75 tests: lib: cmsis_dsp: Disable testing on mps3_an547
This commit disables running the CMSIS-DSP tests on the mps3_an547
board because the QEMU, which is default emulation platform for it,
does not currently support the emulation of the MVE instructions.

Refer to the issue #37694 for more details.

Revert this commit once QEMU 6.2 is released and integrated into the
Zephyr SDK.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2021-08-30 18:17:47 +02:00
Torsten Rasmussen
1cccc8a8fe cmake: increase minimal required version to 3.20.0
Move to CMake 3.20.0.

At the Toolchain WG it was decided to move to CMake 3.20.0.

The main reason for increasing CMake version is better toolchain
support.

Better toolchain support is added in the following CMake versions:
- armclang, CMake 3.15
- Intel oneAPI, CMake 3.20
- IAR, CMake 3.15 and 3.20

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-08-20 09:47:34 +02:00
Stephanos Ioannidis
05fc928a3f tests: benchmark: cmsis_dsp: basicmath: Enable FPU testing
This commit adds a new testcase for the CMSIS-DSP basicmath benchmark
that enables testing with hardware FPU.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2021-08-17 16:41:55 -05:00
Chen Peng1
b676e6a628 tests/benchmarks: add dynamic memory allocation measurement
add a test into latency_measure test case to measure
the average time for dynamic memory allocation and release.

Signed-off-by: Chen Peng1 <peng1.chen@intel.com>
2021-07-21 10:16:56 +03:00
Anas Nashif
11b8dd85b9 boards: minnowboard: remove untested and old board
Remove minnowboard configuration which is very basic and can be brought
back by just taking another X86 configuration. We have not tested this
board for a while and it is not being used actively, so remove it.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-06-01 14:06:56 -05:00
David Brown
22360ee323 tests: benchmarks: mbedtls: Support legacy API
The contents of mbedtls_ecdh_context have changed in newer versions of
the library.  For now, we can work with the old version by adding a
configuration define.  It is unclear how long this will continue to
work.

Signed-off-by: David Brown <david.brown@linaro.org>
2021-05-09 09:59:22 -05:00
Anas Nashif
3b00571160 tests: benchmarks: record benchmark results
Record benchmark results into a CSV file that can be used for tracking.
The data will be available in recording.csv in the build directory.

For example:

cat recording.csv

metric,cycles,nanoseconds
Average thread context switch using yield,11654,11654
Average context switch time between threads (coop),21149,21149
Switch from ISR back to interrupted thread,4928,4927
Time from ISR to executing a different thread,3872,3871
Time to create a thread (without start),4224,4223
Time to start a thread,10784,10783
Time to suspend a thread,10400,10399
Time to resume a thread,10688,10687
Time to abort a thread (not running),1536,1535
Average semaphore signal time,3424,3424
Average semaphore test time,1344,1344
Semaphore take time (context switch),12736,12735
Semaphore give time (context switch),17568,17567
Average time to lock a mutex,1632,1632
Average time to unlock a mutex,4738,4738

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-05-07 23:48:28 -04:00
Jennifer Williams
502d7bd116 tests: benchmarks: remove obsolete boot_time test suite
This test for boot time was sufficient when it was originally
introduced, but is no longer appropriate as the code and
ecosystem grew.

Signed-off-by: Jennifer Williams <jennifer.m.williams@intel.com>
2021-05-05 10:41:15 -04:00
Anas Nashif
80116f1413 tests: benchmarks: footprints: this is not a test
Do not configure this as a test, this will change footprint drastically
and will skew results and tracking.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-04-27 12:15:14 -04:00
David Leach
1a969a026d tests: benchmarks: mbedtls: Conversion of k_work API
Replace all existing deprecated API with the recommended alternative.

Fixes #34107

Signed-off-by: David Leach <david.leach@nxp.com>
2021-04-21 09:39:24 -04:00
Daniel Leung
e3ca3a0930 tests: benchmarks/footprints: update to new kwork API
This updates the footprints app to use the new kwork API.

Fixes #34104

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-04-08 02:48:16 -04:00
Anas Nashif
76f35f29f0 tests: benchmarks: increase timeout
This times out on some platforms, increase timeout a bit to allow slow
platforms to complete.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-03-22 21:39:53 -04:00
Anas Nashif
fe0872c0ab clocks: rename z_tick_get -> sys_clock_tick_get
Do not use z_ for internal APIs, z_ is for private APIs within one
subsystem only.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-03-19 11:22:17 -04:00
Ioannis Glaropoulos
a31bd1539d tests: add missing CONFIG_TEST=y in test suites
In several test suites CONFIG_TEST was missing.
Define CONFIG_TEST=y, so testing-related Kconfig
options (depending on TEST) get switched-on.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2021-03-17 17:48:20 -04:00
Peter Bigot
692bac22f5 tests: footprint: fix thread start races
The test granted access to the user work queue stack from the user
work thread; this was done by k_work_user_queue_start() so was
unnecessary.  Document why it's ok to grant other access after the
work thread was started.

Fix a race condition where the non-work user thread might have started
before it was given access to the resources it needs.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2021-03-08 12:09:16 -05:00
Peter Bigot
4e3b92609b kernel: provide functional equivalent to old userspace work queue API
The new API cannot be used from userspace because it is not merely a
wrapper around existing userspace-capable objects (threads and
queues), but instead requires much more complex and lower-level access
to memory that can't be touched from userspace.  The vast majority of
work queue users are operating from privileged mode, so there's little
motivation to go through the pain and complexity of converting all
functions to system calls.

Copy the necessary pieces of the existing userspace work queue API out
and expose them with new names and types:

* k_work_handler_t becomes k_work_user_handler_t
* k_work becomes k_work_user
* k_work_q becomes k_work_user_q

etc.  Because the replacement API cannot use the same types new API
names are also introduced to make it more clear that the userspace
work queue API is a separate functionality.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2021-03-03 20:06:00 -05:00
Daniel Leung
ce44048d46 x86: rename CONFIG_SSE* to CONFIG_X86_SSE*
This adds X86 keyword to the kconfigs to indicate these are
for x86. The old options are still there marked as
deprecated.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-02-15 08:21:15 -05:00
Daniel Leung
29cf9d879d tests: benchmarks/app_kernel: enable for floating point
The app_kernel benchmarking app has the config file for benchmarking
with floating point enabled, but it was never used. So add it
to the testcase.yaml.

Note that this also limits to run on one CPU on a SMP system as
the resulting numbers would be more consistent among runs.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-02-15 08:21:15 -05:00
Adam Jeliński
9919c14175 m2gl025_miv: Double the test timeouts
The `m2gl025_miv` board needs more time to complete these tests with
Renode.

Signed-off-by: Adam Jeliński <ajelinski@antmicro.com>
2021-02-09 19:41:27 -05:00
Krzysztof Chruscinski
7f08061f0c logging: Revamp menuconfig
Clean up logging menuconfig by grouping configuration into
sections like: mode, processing configuration, backends.

Additionlly, removed LOG_ENABLE_FANCY_OUTPUT_FORMATTING which is no
longer in use.

Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
2021-01-26 06:15:42 -05:00
Martin Åberg
9e463d023f benchmarks: app_kernel: limits to 1 CPU
The app_kernel test hangs or crashes on qemu_x86_64 when more than
one CPU is enabled. So limits the number of CPUs to 1 even when SMP
is enabled.

This issue has probably been masked for some time because the test
was previously marked as being "slow".

Signed-off-by: Martin Åberg <martin.aberg@gaisler.com>
2021-01-15 13:06:33 -05:00
Martin Åberg
4604c45541 tests: enable and run many tests on RISC-V
This commit enables lots of tests on riscv32 and riscv64 which were
previously disabled.

Signed-off-by: Martin Åberg <martin.aberg@gaisler.com>
2021-01-15 13:06:33 -05:00
Hake Huang
2263462aee test: resude sys_kernel loops in small ram
need ram up to 36M for twr_ke18f so reduce the loops

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
2021-01-13 07:38:07 -05:00
Aastha Grover
b644432720 boards: x86: acrn : Add configurations for acrn_ehl_crb
Adding acrn configurations specific to the platform
on which acrn boots zephyr, Only the EHL specifc
configurations for now. Keeping the HW clock frequency to
1900Mhz for EHL and using the new APIc timer driver.

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2021-01-11 16:11:59 -05:00
Daniel Leung
c6253fbe1a tests: latency_measure: fix mis-matched timing start/stop calls
Inside the semaphore tests, there are mis-matched pair of timing
start/stop calls. One called start without calling stop, another
one calling stop twice. So fix them.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-01-04 12:15:30 +01:00
Watson Zeng
7a3f9c4e39 tests: add filter for some tests using newlib
some tests configured with CONFIG_NEWLIB_LIBC=y,
it's better to add a filter filter: TOOLCHAIN_HAS_NEWLIB == 1
in those tests yaml file.

Signed-off-by: Watson Zeng <zhiwei@synopsys.com>
2020-12-16 08:57:40 -05:00
Anas Nashif
22c1228d5b tests: remove posix from allowed archs
This is a simulator platform, remove it from the benchmark.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-09 15:18:29 -05:00
Anas Nashif
dd931f93a2 power: standarize PM Kconfigs and cleanup
- Remove SYS_ prefix
- shorten POWER_MANAGEMENT to just PM
- DEVICE_POWER_MANAGEMENT -> PM_DEVICE

and use PM_ as the prefix for all PM related Kconfigs

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-09 15:18:29 -05:00
Andy Ross
c844bd87b3 kernel: Remove legacy mem_pool usage
The mailbox and msgq utilities had API variants that could pass old
mem_pool blocks through the data structure.  That API is being
deprected (and the features were obscure), so remove the internal
support.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2020-12-07 21:50:14 -05:00
Andy Ross
6965cf526d kernel: Deprecate k_mem_pool APIs
Mark all k_mem_pool APIs deprecated for future code.  Remaining
internal usage now uses equivalent "z_mem_pool" symbols instead.

Fixes #24358

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2020-12-07 21:50:14 -05:00
Kamil Lazowski
a5a3798c6c tests: benchmarks: sys_kernel: Add mem_slab benchmark
Add mem_slab benchmark. It measures times of allocation
and dealloaction of the slabs.

Signed-off-by: Kamil Lazowski <Kamil.Lazowski@nordicsemi.no>
2020-12-07 11:36:48 +01:00
Peter Bigot
1d048df553 tests: add FP formatting to all tests that require it
Tests that include floating-point format specifications may need
cbprintf FP support.  Make sure it's available.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2020-11-17 14:44:59 -06:00
Ningx Zhao
146cf8588b review: rb: develop comments and add some details
Add some comments about rbtree to make it more readable
Add more detail infos to make the purpose and process
of the test cases more clear which include test goal,
test step, input, judging criteria, constraints, etc.,
and these can be seen in our Zephyr documentations.

Signed-off-by: Ningx Zhao <ningx.zhao@intel.com>
2020-11-03 11:08:15 +01:00
Ningx Zhao
9067c1440b tests: initializing rbtree and its node
add 'memset' to initialize rbtree and its node to fix the issue #28175

Signed-off-by: Ningx Zhao <ningx.zhao@intel.com>
2020-10-21 10:06:42 -04:00
Ioannis Glaropoulos
8ef34566ed tests: explicitly disable HW Stack Protection when needed
Some ARM platforms, now, enable HW Stack Protection by
default in the Board definition. So if some tests
need to run without stack protection, it is not
sufficient to disable TEST_HW_STACK_PROTECTION;
we need to explicitly disable HW_STACK_PROTECTION.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2020-09-24 15:42:09 -05:00
Daniel Leung
946c8f0584 tests: latency_measure: move timing_init() earlier
Move the call to timing_init() earlier before function call
to get frequency. Some arch/SoC/board require initialization
before there is a valid frequency value. Or else the printed
value would not be useful.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-09-17 10:54:40 -04:00
Maureen Helm
e839458217 tests: benchmark: Fix footprint testcase.yaml to use arch_allow
Fixes the footprint testcase.yaml to use arch_allow instead of
arch_whitelist.

Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
2020-09-09 15:09:54 -04:00
Anas Nashif
88611de9bd tests: footprint: remove deperecated API call
k_mem_domain_remove_thread is deprecated now.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-09-09 15:09:54 -04:00
Daniel Leung
98fb6e5f22 benchmarks: add an app for commonly used kernel funcs footprint
This adds an app which utilizes common kernel functions as a
starting point to gauge kernel footprint.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-09-09 12:53:10 -04:00
Ningx Zhao
04f8ca38d1 tests: modify tests.benchmarks.datastructure.yaml
Old tags are not clear and difficult to understand,
so modify them and make them more clear.

Signed-off-by: Ningx Zhao <ningx.zhao@intel.com>
2020-09-08 13:51:32 -04:00
Alexandre Bourdiol
4d3580d297 tests: benchmarks: data_structure_perf: rbtree: uninitialized field
Initialize root field to NULL, so that 'test_tree_l.roo.max_depth'
will be assigned a valid value in function rb_insert().

Signed-off-by: Alexandre Bourdiol <alexandre.bourdiol@st.com>
2020-09-08 09:42:27 +02:00
Anas Nashif
6f729383a8 tests: timestamp: minor cleanup
Minor cleanup and fixes.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-09-05 13:28:38 -05:00
Anas Nashif
e90a4bb6b3 tests: latency_measure: Using timing functions
Use new timing API instead if local macros and functions. Add new
becnhmarks for threads and semaphore and change the output to be
parseable.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-09-05 13:28:38 -05:00
Daniel Leung
8197bdea2b tests: timing_info: remove timing info
Remove this benchmark which was relying on custom tracing points in the
code and was not scalable. Use latency_measure benchmark instead which
is more realistic and measures similar metrics in a fully reproducible
manner and on all supported architectures.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-09-05 13:28:38 -05:00
Ningx Zhao
505ea3ee20 tests: updata the rbtree testcase
add a new testcase to:
Verify some operations of rbree are running in
logarithmic time.
Verify an user defined structure contains rbtree node works.
verify "for each"style APIs work.

Signed-off-by: Ningx Zhao <ningx.zhao@intel.com>
2020-09-04 08:00:05 -04:00
Ningx Zhao
9ef5a809dd tests: verify the time complexity of dlist
Add two test cases to verify the operations of accessing
head,tail,insert and remove in constant time by proving the time
complexity of the operations are O(1).

Signed-off-by: Ningx Zhao <ningx.zhao@intel.com>
2020-09-04 11:58:52 +02:00
Tomasz Bursztyka
4b9134d8d2 tests: Apply IRQ offload API change
Switching to constant parameter.

Fixes #27399

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2020-09-02 13:48:13 +02:00
Anas Nashif
dca317c730 sanitycheck: inclusive language
change whitelist -> allow.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-08-27 07:04:07 -04:00