Commit Graph

70 Commits

Author SHA1 Message Date
Anas Nashif
d4e03991a0 ci: have twister cleanup artifacts as it goes
Avoid running out of space by cleaning up passing tests.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-05-05 18:48:45 -05:00
Anas Nashif
e366d7948a ci: remove bsim from buildkit execution
We now have a dedicated action for this.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-04-23 21:40:10 -04:00
Carles Cufi
e9e28c0562 ci: Re-enable bsim in CI
BabbleSim execution was (AFAICT mistakenly) disabled in
dda6a5ee90d0f5b31e7d269761dc2ab1a2f8510c. Re-enable it so we run them in
CI again.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2021-03-11 08:52:18 -05:00
Kumar Gala
eea17e5441 ci: scripts/ci/run_ci: Cleanup file names
Rename test_file_[1..4] to something a bit more meaningful.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2021-03-08 08:35:56 -06:00
Anas Nashif
82daea910b ci: limit CI to only changed architecture
If files are changed for architecture code, then only run CI on those
architectures.

Combine all scripts into one and make it simpler.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-03-04 07:14:11 -06:00
Anas Nashif
3572b518f6 ci: remove shippable references
We now use buildkite.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-03-04 07:14:11 -06:00
Anas Nashif
b2a5782570 ci: remove west extension call
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-08 07:49:19 -05:00
Anas Nashif
f1dea627cc ci: fetch sof submodules
Fetch submodules needed to build with SOF.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-06 07:53:46 -06:00
Anas Nashif
a05d056943 ci: rename sanitycheck -> twister
Change all CI scripts to use the new script name: twister.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Kumar Gala
52ad9a468c ci: run_ci.sh: retry failed west update for PR builds
We made a similar change for the nightly builds, but the PR builds
invoke run_ci.sh, so we need the same change here.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-11-23 14:42:54 -05:00
Anas Nashif
bb2d899c32 ci: run module tests only on major changes
Do not run module tests if core code isn't being changed.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-09-03 18:11:19 +02:00
Anas Nashif
8b5e166f8e ci: run_ci: do not call buildkite-agent
Do not call buildkite here, this script should be callable to reproduce
CI results.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-09-03 18:11:19 +02:00
Kumar Gala
33056e48c5 ci: make test_file.txt a build artifact for debug
Rather that echo'ng the test_file.txt to the console, lets just upload
it as a buildkite artifact.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-07-30 11:25:25 -04:00
Anas Nashif
314b8af797 ci: enable sanitycheck for integration
Enable sample for integration.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-30 08:00:03 -04:00
Kumar Gala
feeda05323 ci: Add a debug out of what tests we are going to build
Add a cat of the test_file.txt so we can see exactly what tests are
expected to be built and run.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-07-24 12:10:15 +02:00
Anas Nashif
33c21e1f7d ci: do not skip BT simulator tests
BT simulator tests run standalone and independent of sanitycheck and
might be triggered by tests that are not managed by sanitycheck, so
leave it alone.

Fixes #26508

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-06-29 12:16:37 -04:00
Kumar Gala
491bbad8e8 ci: Update for buildkite environment
* Tweak west_setup:
  - log `west update` to a file to reduce noise in log
  - use `west forall` + `git reset` to make sure files are checked out
    (this is to handle a possible module cache)
* Output when we start sanity_check.  Add a banner for when we run
  sanity_check so it's a bit easier to find in console logs

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-06-09 15:28:48 -04:00
Anas Nashif
29c0c08fd4 ci: do full sanitycheck if not a PR
Cover the cases where we run sanitycheck on a merged PR.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-04-01 17:20:56 -04:00
Anas Nashif
270165428c ci: do not run full sanitycheck if not needed...
if we are changing a test or a sample, there is not need to run
everything, it is enough to just build/run the changed test/sample.

Also, some files have impact on the code and if they are being changed,
do not run fully sanitycheck.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-04-01 14:00:57 -04:00
Anas Nashif
c7092a1c4d ci: when merging files, keep one header row
When merging files, make sure to remove header rows from each file and
just keep one header. This will stop us from parsing header rows as
tests.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-03-24 12:15:36 -04:00
Kumar Gala
2fd7876aa7 ci: github: west: Add a GH workflow to run the west command tests
Replace running west command tests in run_ci.sh with a github workflow.
This provides some benefits in that we can run the west command tests on
multiple python versions and host OSes (linux, mac and windows).

Also have the benefit that the tests are only run on modifications to
files in scripts/west_commands/ or scripts/west-commands.yml.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-02-06 08:34:43 -06:00
Alberto Escolar Piedras
1a01a10a87 ci: Run also BT edtt_ble_test_app tests
Include in the CI run the EDTT tests

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2020-01-31 18:36:40 +01:00
Alberto Escolar Piedras
9338e9215b run_ci.sh: Set EDTT_PATH
So that the EDTT tool could be used by this script,
set its path so other scripts could find it

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2020-01-31 18:36:40 +01:00
Anas Nashif
d6d0b6454b ci: do not display timestamps
With timestamps enabled, the log in shippable is very difficult to read,
so remove those and enable them when needed for debugging only.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-17 08:30:23 -06:00
Anas Nashif
81b2f37a9d ci: run sanitycheck in verbose mode
Call sanitycheck in verbose mode

Fixes #21483

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-18 12:07:49 -05:00
Anas Nashif
36e1fb630b ci: run sanitycheck on tests and samples
If modules announce they have tests, run sanitycheck on those modules.
This will run on first matrix node and as the last step after running
regular testsuite.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-16 18:56:28 +01:00
Alberto Escolar Piedras
dfed64c2a4 nrf52_bsim: Use HW models from new west module
Fetch HW models from a new west module.
And, remove all pre-west glue which was used to:
 * Fetch them in CI
 * Validate their vesion
 * Modify the include path and link to them

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2019-12-11 11:33:02 +01:00
Ulf Magnusson
94d1db03af ci: run_ci: Remove semicolons at ends of lines
Not needed in sh.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-12-10 14:34:46 -08:00
Ulf Magnusson
85625aa27f ci: run_ci: Use lowercase for internal vars
This makes it possible to tell at a glance which variables are internal
to the script and which ones are parameters to it, which is very
helpful.

This convention is pretty common. See e.g. Google's shell style guide at
https://google.github.io/styleguide/shell.xml#Naming_Conventions, and
https://github.com/icy/bash-coding-style#naming-and-styles. It's older
than those though.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-12-10 14:34:46 -08:00
Anas Nashif
33e6384e48 ci: cleanup test manifests
Make sure we cleanup the manifest files at the right spot of the CI
process. When not building a PR, behavior is different.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-10-11 20:05:03 +02:00
Anas Nashif
2f0be4334b ci: adapt ci script for new sanitycheck
We now run sanitycheck with different options, so adapt script.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-10-11 07:28:19 -07:00
Kumar Gala
127a245fb0 ci: run_ci: Refactor west setup into a function
Move west setup into its own function.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-09-18 19:13:37 -05:00
Kumar Gala
87c81b7547 ci: run_ci: merge two if blocks
The same if check is done twice back to back, we can merge into a single
if block.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-09-18 19:13:37 -05:00
Kumar Gala
020f87eae8 ci: run_ci: Move functions
Move functions up to make the code a bit more readable.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-09-18 19:13:37 -05:00
Kumar Gala
e717d45ac1 ci: run_ci: handle if the git dir isn't named zephyr
For cases like zephyr-test or for other reasons the repo dir might not
be called zephyr.  Just use the basename of the dir instead of assuming
its called zephyr.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-09-18 19:13:37 -05:00
Kumar Gala
3209cad030 ci: run_ci: Remove stale DOC_MATRIX variable
The script doesn't use DOC_MATRIX so remove this stale variable.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-09-18 19:13:37 -05:00
Anas Nashif
4132ee3a05 ci: do not export BSIM env vars if bsim is not available
When running run_ci.sh locally with:

 ./scripts/ci/run_ci.sh -l -b master -R upstream/master..

we export BSIM_OUT_PATH unconditionally which causes sanitycheck to fail
on nrf52_bsim (it depends on BSIM_OUT_PATH variable).

Check if the path defined in BSIM_OUT_PATH is available and unset env.
variable if it is not.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-09-07 09:04:40 -04:00
Marti Bolivar
2b3d9df1d7 west: require v0.6.0 or higher
The main change is the elimination of the bootstrapper, a design flaw
/ misfeature.

Update the documentation to be compatible with the 0.6.x releases as
well. This has to be done atomically, as there were incompatible
changes.  Make use of the versionchanged and versionadded directives
to begin keeping track of how these APIs are evolving.

(Note that west 0.6.0 will remain compatible with the extension
commands in Zephyr v1.14 LTS as long as that is still alive. This
change is targeted towards Zephyr 2.0 users.)

This requires a bump in the shippable container and allows us to
simplify the west_commands test procedure.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-08-05 15:17:45 +02:00
Anas Nashif
cbfba62182 ci: increase sleep time between sanitycheck runs
Give the system more time to settle before we try to re-run the tests
again.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-06-20 22:15:27 +02:00
Anas Nashif
f2cb20c772 docs: fix misspelling across the tree
Found a few annoying typos and figured I better run script and
fix anything it can find, here are the results...

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-06-19 15:34:13 -05:00
Anas Nashif
b58aa20e13 ci: skip handling coverage data
We not analysing coverage data at all in CI right now. Disable this
while we figure out a better solution for reporting data.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-06-14 09:26:40 -04:00
Anas Nashif
3ae52624ff license: cleanup: add SPDX Apache-2.0 license identifier
Update the files which contain no license information with the
'Apache-2.0' SPDX license identifier.  Many source files in the tree are
missing licensing information, which makes it harder for compliance
tools to determine the correct license.

By default all files without license information are under the default
license of Zephyr, which is Apache version 2.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-07 08:45:22 -04:00
Anas Nashif
193fb971c2 ci: do not report coverage on pull requests
The code coverage reports on pull requests has been inconsistent and
confusing. Disable them for pull-requests.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-01 18:39:17 -04:00
Anas Nashif
c9d1bb90d8 ci: add option to specify custom range
This is useful when running the script locally on a set of commits to
reproduce how things are run in CI.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-03-04 14:08:12 -05:00
Marc Herbert
6b4cda9b25 run_ci.sh: git log what [base] version is being tested
Zephyr Continuous Integration made the choice of rebasing pull requests
on its fast moving master branch before testing them. Unfortunately the
Shippable interface assumes the other choice[*] and highlights the SHA1
of the pull request _before_ rebasing which is very misleading.  For now
the only way to realize this disconnect is to scrutinize the build logs
and/or this source code (or tribal knowledge).

To mitigate and reduce a bit this confusion, run a short:
  git log -n 5 --oneline --decorate
before and after rebase.

Making it worse, Shippable sub-runs are not pegged to a common base
version. For instance the reason why shippable sub-run 35578.1 passed
while all other 35578.x sub runs failed is very likely because PR #13803
was merged to master in the mean time. This commit would have been able
to prove that.

[*] In an ideal world with infinite time and resources, Continuous
Integration would test pull requests BOTH alone and combined with the
moving tip.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-03-01 09:39:31 +01:00
Marc Herbert
f7e995c075 run_ci.sh: run pytest-3 if found
On (at least) Fedora "pytest" is python2 and python2 only.

Also update remote name in sample invocation from "upstream" to "origin"
because that's how west sets things up by default (and this script
requires west)

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-03-01 09:39:31 +01:00
Anas Nashif
12f737163f ci: run_ci.sh: make script run locally
Allow running script locally. For example, on a local tree with local
changes:

./scripts/ci/run_ci.sh -b master -r upstream  -l

This will be have the same way as in CI and creates a test manifest
based on changes.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-02-01 19:04:55 -05:00
Anas Nashif
8a51a4bf26 ci: do not abort if west is already initialised
We should not abort if west was initiaized, someone must have
initialized, i.e. locally and everything is already setup for us and
ready to go.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-02-01 19:04:55 -05:00
Marti Bolivar
63841df184 scripts: run_ci.sh: add pytest-based testing for west commands
Incorporate these into run_ci.sh the same way that btsim results were
done. This adds a dependency on pytest.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-01-29 10:15:01 +01:00
Marti Bolivar
9cecb5df4a scripts: run_ci.sh: remove dead code
Commit 88ece494 ("ci: do not build docs in main CI job") removed the
call to build_docs but not the function itself. That makes it
confusing to try to find where it's called; remove it.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-01-29 10:15:01 +01:00