Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Development: Enable parallel server test execution #6796

Merged
merged 105 commits into from
Sep 22, 2023

Conversation

DominikRemo
Copy link
Contributor

@DominikRemo DominikRemo commented Jun 30, 2023

Checklist

General

Server

Motivation and Context

Fast tests 🏎️
Improves server-test execution time from up to 30 min 🚗 => down to 12 min 🏎️

Description

Enables parallel testing with JUnit 5 JUnit Docs: 2.20. Parallel Execution.

Implements a new JUnit 5 extension and custom Logback appender for logging in parallel tests.

Fixes several flaky tests (e.g.: in classes LongFeedbackResourceIntegrationTest, NotificationScheduleServiceTest and ProgrammingExerciseScheduleServiceTest)

Re-organized tests into different groups for better balancing.

  • We confirm that the tests on this branch are at least as reliable as on the develop branch. We tested the reliability of the tests with a custom build plan (development-enable-parallel-unit-testing on https://bamboo.ase.in.tum.de/browse/ARTEMIS-STO0).
    We used a custom plan that only executes the server tests. Besides only running the server tests, the Server Tests Only plan is similar to the Artemis Test plan

Steps for Testing

Test changes only

Review Progress

Code Review

  • Code Review 1
  • Code Review 2

@DominikRemo DominikRemo changed the title Development: Enable parallel Unit Testing Development: Enable parallel Testing Jul 6, 2023
@DominikRemo DominikRemo changed the title Development: Enable parallel Testing Development: Enable parallel test execution Jul 6, 2023
Note: Since this change affects the test execution order, some tests might fail. I commit this for asserting that the tests actually run (fail) faster with parallel execution enabled on bamboo and github actions. The tests will be fixed in another commit
@ls1intum ls1intum deleted a comment from codecov-commenter Jul 14, 2023
@DominikRemo DominikRemo removed the small label Jul 24, 2023
Strohgelaender
Strohgelaender previously approved these changes Sep 21, 2023
Copy link
Contributor

@Strohgelaender Strohgelaender left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code lgtm, tests are much faster now 🚀 thanks for implemeting!

Copy link
Contributor

@terlan98 terlan98 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well done! The tests are indeed a lot faster now 🚀
The code looks good, but I have one question:

terlan98
terlan98 previously approved these changes Sep 21, 2023
Copy link
Collaborator

@MaximilianAnzinger MaximilianAnzinger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for considering my comment. Hopefully, this gets merged asap 👍

terlan98
terlan98 previously approved these changes Sep 21, 2023
Strohgelaender
Strohgelaender previously approved these changes Sep 22, 2023
@github-actions github-actions bot added the client Pull requests that update TypeScript code. (Added Automatically!) label Sep 22, 2023
@krusche krusche changed the title Development: Enable parallel test execution Development: Enable parallel server test execution Sep 22, 2023
@krusche krusche merged commit 5ba2676 into develop Sep 22, 2023
15 of 19 checks passed
@krusche krusche deleted the development/enable-parallel-unit-testing branch September 22, 2023 20:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
client Pull requests that update TypeScript code. (Added Automatically!) ready to merge server Pull requests that update Java code. (Added Automatically!) tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants