Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: added nightly run from spread-cron #14729

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

maykathm
Copy link
Contributor

@maykathm maykathm commented Nov 15, 2024

This PR migrates all cron jobs from spread-cron to snapd that run spread tests on snapd. It makes a minor change in the spread-test.yaml to allow for use of the master snap along with two required environment variables.

It also adds the possibility of executing with (in addition to manually specifying the systems) "ALL" systems or "NONE." To help not create confusion, it will no longer be possible to leave the systems input blank.

To ensure the commands will run correctly, I ran a job in snapd-ci with cron disabled to check the executed spread commands. To ensure the scheduling mechanism works correctly, I pushed the changes to the master branch of snapd-ci and saw the three triggers 2 am UTC for nested and nightly, 3 am UTC for experimental and openstack, and 6 am UTC for build from current.

Cron jobs

spread-cron - snapd-nightly-suite

Command in spread-cron

spread -v google:tests/nightly/

Command in new nightly tests

spread -no-debug-output -logs spread-logs google:tests/nightly/...

spread-cron - snapd-test-build-from-current

Command in spread-cron

spread -v google:ubuntu-18.04-64:tests/... \
          google:ubuntu-20.04-64:tests/... \
          google:ubuntu-22.04-64:tests/... \
          google:ubuntu-24.04-64:tests/... \
          google-distro-1:debian-11-64:tests/... \
          google-distro-1:debian-12-64:tests/... \
          google-distro-1:debian-sid-64:tests/...

Command in new nightly tests

spread -no-debug-output -logs spread-logs google:ubuntu-18.04-64:tests/... google:ubuntu-20.04-64:tests/... google:ubuntu-22.04-64:tests/... google:ubuntu-24.04-64:tests/...

and

spread -no-debug-output -logs spread-logs google-distro-1:debian-11-64:tests/... google-distro-1:debian-12-64:tests/... google-distro-1:debian-sid-64:tests/...

spread-cron - snapd-test-experimental

Command in spread-cron

spread -v google:ubuntu-18.04-64:tests/ google:ubuntu-20.04-64:tests/ google:ubuntu-21.10-64:tests/ google:ubuntu-core-20-64:tests/ 

Command in new nightly tests

spread -no-debug-output -logs spread-logs google:ubuntu-18.04-64:tests/... google:ubuntu-20.04-64:tests/... google:ubuntu-21.10-64:tests/... google:ubuntu-core-20-64:tests/...

spread-cron - snapd-test-openstack

Command in spread-cron

spread -no-debug-output openstack:tests/main/

Command in new nightly tests

Currently not available because the testing environment doesn't have spread-plus. The output will be: spread -no-debug-output -logs spread-logs openstack:tests/main/...

Copy link

codecov bot commented Nov 15, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.03%. Comparing base (96ea7b0) to head (9cfade7).
Report is 111 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #14729      +/-   ##
==========================================
+ Coverage   78.95%   79.03%   +0.08%     
==========================================
  Files        1084     1087       +3     
  Lines      146638   147906    +1268     
==========================================
+ Hits       115773   116898    +1125     
- Misses      23667    23769     +102     
- Partials     7198     7239      +41     
Flag Coverage Δ
unittests 79.03% <ø> (+0.08%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

.github/workflows/nightly-spread.yaml Outdated Show resolved Hide resolved
.github/workflows/nightly-spread.yaml Outdated Show resolved Hide resolved
.github/workflows/nightly-spread.yaml Show resolved Hide resolved
.github/workflows/nightly-spread.yaml Show resolved Hide resolved
tests/lib/spread/rules/nightly.yaml Outdated Show resolved Hide resolved
@maykathm maykathm force-pushed the SNAPDENG-34131-migrate-nightly-tests-from-cron-to-snapd branch 8 times, most recently from 795d9a0 to def93ae Compare November 19, 2024 20:07
.github/workflows/spread-tests.yaml Outdated Show resolved Hide resolved
.github/workflows/spread-tests.yaml Outdated Show resolved Hide resolved
.github/workflows/spread-tests.yaml Outdated Show resolved Hide resolved
.github/workflows/spread-tests.yaml Outdated Show resolved Hide resolved
.github/workflows/spread-tests.yaml Outdated Show resolved Hide resolved
tests/lib/spread/rules/nightly.yaml Outdated Show resolved Hide resolved
tests/lib/spread/rules/nightly.yaml Outdated Show resolved Hide resolved
@maykathm maykathm force-pushed the SNAPDENG-34131-migrate-nightly-tests-from-cron-to-snapd branch from 0d40ace to b0a65c3 Compare November 21, 2024 18:32
Copy link
Contributor

@zyga zyga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partial review, I'll do more tomorrow.

.github/workflows/nightly-spread.yaml Show resolved Hide resolved
.github/workflows/spread-tests.yaml Outdated Show resolved Hide resolved
@zyga zyga self-requested a review November 27, 2024 13:26
Copy link
Collaborator

@sergiocazzolato sergiocazzolato left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

…ap usage and check for empty rules when running spread
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants