If CONFIG_SMP_TEST_RUN_FACTOR is zero, the switch torture test
is effectively not doing anything as the k_sleep() below is not
going to sleep at all, and all created threads are being
terminated (almost) immediately after creation. So if run
factor is zero, mark the test as skipped.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Test case for threads abort issue.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
SMP tests `inc_concurrency` and `smp_switch_torture` use a 'stressing'
approach to verify their results: run something for some time (or some
number of repetitions). However, in some environments, current 'stress'
levels can be quite high, making tests take a long time - environments
like emulators/simulators.
This patch adds a Kconfig that allows one to define a percentage factor
to the 'stress' (time or repetitions) used by these tests.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
For some kernel tests, faults and exceptions are expected.
They are caught and the test would continue if the reasons
for faults are as expected. However, when the unexpected
reasons are encountered, the code simply prints a message
and calls k_fatal_halt(). When running under twister,
these messages are not the expected failed messages so
twister will spin till timeout although the execution
has already been halted. This adds another printk() before
halt to signal twister that the test has failed and bails
early.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
After the dbe3874079 - (tests: kernel/smp: wait for threads to exits
between tests) I've started seeing sporadic kernel.multiprocessing.smp
test failures on our platforms.
------------------------------->8---------------------------------
[*snip*]
===================================================================
START - test_fatal_on_smp
E: r0: 0x3 r1: 0x0 r2: 0x0 r3: 0x0
E: r4: 0x80000194 r5: 0x0 r6: 0x0 r7: 0x0
E: r8: 0x800079c4 r9: 0x82802 r10: 0x80008d8c r11: 0x8000dad8
E: r0: 0x3 r1: 0x2712 r2: 0x114 r3: 0x0
E: r4: 0xf4240000 r5: 0x0 r6: 0xf424 r7: 0xbe40
E: r8: 0x2540 r9: 0x0 r10: 0x80008d8c r11: 0x8000db8c
E: r12: 0x8000ddf0 r13: 0x0 pc: 0x80000aec
E: blink: 0x80000ae6 status32: 0x80082002
E: >>> ZEPHYR FATAL ERROR 3: Kernel oops on CPU 0
E: Current thread: 0x8000db8c (test_fatal_on_smp)
E: r12: 0x8000ddf0 r13: 0x0 pc: 0x8000019a
PASS - test_fatal_on_smp in 0.014 seconds
===================================================================
START - test_get_cpu
E: blink: 0x80001490 status32: 0x80082002
E: >>> ZEPHYR FATAL ERROR 3: Kernel oops on CPU 1
E: Current thread: 0x8000dad8 (unknown)
------------------------------->8---------------------------------
The rootcause if that we doesn't proper cleanup resources after
test_fatal_on_smp test case. So child thread we start test_fatal_on_smp
may continue running for some time after the test_fatal_on_smp
test case is finished.
As in the next test case (test_get_cpu) we use same thead structures
again to create new child thread we may actually rewrite some data of
thread which is still running (or vise versa).
As we trigger the crash in test_fatal_on_smp we can't simply join
child thread in the end of test case (as we never get here). We can't
simply use join child thread before we initiate crash in test_fatal_on_smp
either as we don't want to introduce reschedule point here which may break
the test logic.
So, to fix that, we'll just do k_busy_wait in test_fatal_on_smp
thread after we start child thread to wait for thread trigger
exception and being terminated.
To verify that we also assert that child thread is dead by the
time when we stop busy waiting.
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Move runtime checks to use arch_num_cpus() and build checks
to use CONFIG_MP_MAX_NUM_CPUS. This is to allow runtime
determination of the number of CPUs in the future.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
The k_sys_fatal_error_handler() function is declared in zephyr/fatal.h
with a name of "esf" for the second parameter, not "pEsf". For
unknown reasons, this is showing up in CI as a documentation
generation failure pointing at the (correct) header.
Still, no reason not to synchronize.
Signed-off-by: Andy Ross <andyross@google.com>
Obviously the test of the feature won't work if we don't have an IPI.
And there were two threads that spawned threads that enter busy loops,
expecting to be able to abort them from another CPU. That doesn't
work[1] without an IPI. Just skip these cases rather than trying to
kludge up some kind of abort signal.
[1] Rather, it does work, it just takes infinite time for these
particular test cases. Eventually the CPU would be expected to
receive some other interrupt like a timeout, which would work to
abort the running thread. But no such timer was registered.
Signed-off-by: Andy Ross <andyross@google.com>
Inside test_get_cpu, the current CPU ID is stored in the test
thread's stack. Another thread is spawned with a pointer to
the variable holding this CPU ID, where this thread is supposed
to run on another CPU. On a cache incoherent platform, this
value of this variable may not have been updated on other CPU's
internal cache. Therefore when checking CPU IDs inside the newly
spawned thread, it is not checking the passed in CPU ID, but
actually whatever is on the another CPU's cache. This results in
random failure on the test_get_cpu test. Since for cache
incoherence architectures, CONFIG_KERNEL_COHERENCE is enabled by
default on SMP where shared data is placed in multiprocessor
coherent (generally "uncached") memory. The fix to this is to
simply make this variable as a global variable, as global
variable are consided shared data and will be placed in
multiprocessor coherent memory, and thus the correct value will
be referenced inside the newly spawned thread.
Fixes#49442
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds a bunch of k_thread_join() to make sure threads spawned
for a test are no longer running between exiting that test. This
prevents interference between tests if some threads are still
running when assumed not.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Add a bunch of missing "zephyr/" prefixes to #include statements in
various test and test framework files.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Files including <zephyr/kernel.h> do not have to include
<zephyr/zephyr.h>, a shim to <zephyr/kernel.h>.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
In order to bring consistency in-tree, migrate all tests to the new
prefix <zephyr/...>. Note that the conversion has been scripted, refer
to #45388 for more details.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
This test case was taking a (traditional) irq_lock(), which masks
interrupts, and then calling k_mutex_lock() with a timeout of
K_FOREVER, which is a blocking call. That's not legal, because it
will obviously schedule other threads to run in a context where the
code was promised it would not. This used to be an uncaught error,
but now we have an assertion that catches this.
It's not clear what this test case is supposed to be testing, as the
behavior is actually identical to the release_global_lock case except
for the (incorrect) addition of the irq_lock(). If this is needed for
code coverage we can work to figure out the real root cause of the
missing coverage later.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
A few mistakes in recent changes to this test:
There was a "LOCK_NO" (i.e. no locking!) case being exercised in
test_inc_concurrency, where three threads would race against each
other incrementing and decrementing a single count without
synchronization. And... it failed on cAVS. Because there was no
synchronization. Just remove.
The LOCK_IRQ (irq_un/lock()) case of the same test was was casting
taking a pointer to an integer (that stored the irq_lock() result) and
casting the pointer value to an integer instead of dereferencing it.
Also the workq test had a work item on the stack, which is forbidden
when KERNEL_COHERENCE=y
Fixes#34152
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Update testcase test_fatal_on_smp(), and refine it and correct some
inappropriate usage such as unnecessary irq_lock(). This prevents
the error propagation to the later executing testcase.
Fixes#35200Fixes#35202
Signed-off-by: Enjia Mai <enjiax.mai@intel.com>
This PR add 2 module test cases:
- test_smp_release_global_lock() and test_smp_release_global_lock_irq()
verify z_smp_release_global_lock() works.
And 1 integration test cases:
- test_inc_concurrency() to verify parallelly increase operations will
fail if not applying synchronization on SMP.
Signed-off-by: Enjia Mai <enjiax.mai@intel.com>
This is an integration testcase for smp. It tests
the situation when smp is configed. Fatal can be invoked on
different core and system workq can also be run on different
core.
Signed-off-by: Ying ming <mingx.ying@intel.com>
Do minor change of the descriptions and doxygen group name in order to
pave the way for generation the test specification.
Signed-off-by: Enjia Mai <enjiax.mai@intel.com>
Add more detail description for test case of arch_curr_cpu() and
arch_sched_ipi(). This is in order to make the purpose and process of
the test cases more clear.
Signed-off-by: Enjia Mai <enjiax.mai@intel.com>
This test was written to assume that the only IPI handled would be the
one generated by the test, but the scheduler also generates an IPI any
time a thread becomes runnable, and there's no way to lock that out in
an SMP system where the other CPU is going to be doing its own thing
(we can't use "1cpu" because that locks interrupts on the other CPU
and obviously this is a test of an interrupt).
Change the logic to detect that "at least one IPI was received", which
is fine for coverage. Really a better place for a test like this
would have been tests/kernel/mp, which is a test of the lower level
APIs and runs the other CPU deterministically (i.e. not under the
control of the Zephyr scheduler).
Also some misc fixes:
* Don't busy wait at the start, that's needless.
* Sleep instead of busywaiting after sending the IPI, spinning isn't
needed here and acts to increase CI load needlessly.
* Declare the cross thread signal variable volatile for correctness
(though this error seems to have been benign in practice).
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Add one another test case for testing both arch_curr_cpu() and
arch_sched_ipi() architecture layer interface.
Signed-off-by: Enjia Mai <enjiax.mai@intel.com>
Sleeping for a full second at startup is needless. The currently
enabled subsystems on platforms that run this test don't even have any
other threads running at startup, so we're guaranteed the other core
is in idle before we even reach main(). Just a few ms is plenty.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Kernel timeouts have always been a 32 bit integer despite the
existence of generation macros, and existing code has been
inconsistent about using them. Upcoming commits are going to make the
timeout arguments opaque, so fix things up to be rigorously correct.
Changes include:
+ Adding a K_TIMEOUT_EQ() macro for code that needs to compare timeout
values for equality (e.g. with K_FOREVER or K_NO_WAIT).
+ Adding a k_msleep() synonym for k_sleep() which can continue to take
integral arguments as k_sleep() moves away to timeout arguments.
+ Pervasively using the K_MSEC(), K_SECONDS(), et. al. macros to
generate timeout arguments.
+ Removing the usage of K_NO_WAIT as the final argument to
K_THREAD_DEFINE(). This is just a count of milliseconds and we need
to use a zero.
This patch include no logic changes and should not affect generated
code at all.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
This test was whiteboxing the _current_cpu pointer to extract the CPU
ID. That's actually racy: the thread can be preempted and migrated to
another CPU between the _current_cpu expression and the read of the ID
field. Do it right.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Promote the private z_arch_* namespace, which specifies
the interface between the core kernel and the
architecture code, to a new top-level namespace named
arch_*.
This allows our documentation generation to create
online documentation for this set of interfaces,
and this set of interfaces is worth treating in a
more formal way anyway.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Re-run with updated script to convert integer literal delay arguments to
k_sleep to use the standard timeout macros.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Just housekeeping around the casting between void * arguments to
thread functions and integer types.
Signed-off-by: Charles E. Youse <charles.youse@intel.com>
For obvious performance reasons, scheduler state changes (other than
aborting a thread) do not cause synchronous interrupts on the other
CPU. Doing a k_thread_wakeup() means that the current CPU will run it
synchronously if it's high priority, but if you want to see it run on
the other cores you need to wait for them to reach a scheduling point
on their own.
The test was written to assume that k_thread_wakeup() is synchronous,
but that's not right, and it needs to spin a bit. This bug was always
present in the test, but masked by a bug in the way that k_sleep() was
handled on SMP. See #9506.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
There was a missing 'z_' renaming to
z_is_thread_prevented_from_running which would have caused
sanitycheck to fail but it is not being built at the moment.
Fix this first.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The various tests would all do a "wait for threads to exit" step
before checking the results, but this was implemented with a simple
busy wait that turns out to need careful tuning (because there was
busy waiting in the threads).
Rather than try to synchronize this, white box the issue (it's a low
level SMP test, after all) by spinning on the thread states directly
watching for the kernel to flag them dead. The downside here is that
if the process fails for some reason we'll get a hang and a timeout
reported from sanitycheck and not a synchronous ztest assertion. But
in return, successful tests run much faster and I don't need to worry
about how to tune them for IPI latency on different platforms.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
This case was predicated on a mistake. The behavior of k_wakeup() has
always been NOT to wake up threads that are "pending" on a wait queue,
only ones blocked on a timeout in k_sleep(). As written, this test
case could never pass.
(Really there's no good reason for that. It seems reasonable to me to
expect wakeup to work symmetrically, and the docs are sort of
ambiguous on the subject. But the code in k_wakeup() is clear:
threads flagged pending get an early exit and the call becomes a
noop.)
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
There was a test-created thread that wasn't including this. It's a
huge stack and doesn't overflow (though I thought briefly that it
was), but it's a rule that we need to have that buffer and I'm trying
to fix these as I find them.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Update reserved function names starting with one underscore, replacing
them as follows:
'_k_' with 'z_'
'_K_' with 'Z_'
'_handler_' with 'z_handl_'
'_Cstart' with 'z_cstart'
'_Swap' with 'z_swap'
This renaming is done on both global and those static function names
in kernel/include and include/. Other static function names in kernel/
are renamed by removing the leading underscore. Other function names
not starting with any prefix listed above are renamed starting with
a 'z_' or 'Z_' prefix.
Function names starting with two or three leading underscores are not
automatcally renamed since these names will collide with the variants
with two or three leading underscores.
Various generator scripts have also been updated as well as perf,
linker and usb files. These are
drivers/serial/uart_handlers.c
include/linker/kobject-text.ld
kernel/include/syscall_handler.h
scripts/gen_kobject_list.py
scripts/gen_syscall_header.py
Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
The idea behind this test is to race two CPUs against each other,
validating that each is truly running simultaneously. But it just
assumed that the other thread/cpu would start synchronously as soon as
k_thread_create() returned.
Normally, that works fine. But while debugging I added some code that
was slowing down entry to the other thread (or maybe the return from
k_thread_create() into the main thread) long enough to allow one
thread to get a significant head start. That breaks the logic in the
test and things were inexplicably "failing".
Put a spin loop around the count so that the main thread can start
counting to within the memory system's ability to inform it of the
change from the other thread.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Simple SMP test to validate the two threads can be simultaneously
scheduled. Arranges things such that both threads are at different
priorities and never yield the CPU, so on a uniprocessor build they
cannot be fairly scheduled. Checks that both are nonetheless making
progress.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
