Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Development: Fix flaky PostIntegrationTest #7205

Merged
merged 8 commits into from
Sep 22, 2023

Conversation

laadvo
Copy link
Contributor

@laadvo laadvo commented Sep 15, 2023

Checklist

General

Server

Motivation and Context

The test PostIntegrationTest.testGetPostsForCourse_WithExerciseIdRequestParam() is flaky and sometimes fails with

org.mockito.exceptions.misusing.UnfinishedStubbingException: org.mockito.exceptions.misusing.UnfinishedStubbingException: 
Unfinished stubbing detected here:
-> at de.tum.in.www1.artemis.util.AbstractArtemisIntegrationTest.mockMailService(AbstractArtemisIntegrationTest.java:129)

E.g. thenReturn() may be missing.
Examples of correct stubbing:
    when(mock.isOk()).thenReturn(true);
    when(mock.isOk()).thenThrow(exception);

This PR fixes the test testCreateAnnouncement() which is executed before testGetPostsForCourse_WithExerciseIdRequestParam() and is causing it to be fail.

Description

In the test PostIntegrationTest.testCreateAnnouncement() mails are sent using the javaMailSender in async. The mails sometimes get sent after the javaMailSender already mock got reset. When the mock is configured again in the @BeforeEach method mockMailService() there already exists a recorded call on the mock which causes the cryptic UnfinishedStubbingException.
This happened in the @BeforeEach of testGetPostsForCourse_WithExerciseIdRequestParam() which is always run directly after testCreateAnnouncement().

The PR adds a verify statement at the end of testCreateAnnouncement(), making sure that the mails are sent before starting the next test.

Review Progress

Look at the code changes and feel free to run PostIntegrationTest multiple times using the following script (inspired by the script from #6506).

rm postIntegration.log
for i in {1..50}
do
   echo "Starting run: i=$i."
   ./gradlew cleanTest test --tests de.tum.in.www1.artemis.metis.PostIntegrationTest -x webapp | tee -a postIntegration.log
done

echo "Print failed tests:"
cat postIntegration.log | grep "testGetPostsForCourse_WithExerciseIdRequestParam() FAILED" || [[ $? == 1 ]]

Code Review

  • Code Review 1
  • Code Review 2

Test Coverage

unchanged

@laadvo laadvo changed the title Development: Fix PostIntegrationTest Development: Fix flaky PostIntegrationTest Sep 18, 2023
@laadvo laadvo marked this pull request as ready for review September 19, 2023 13:02
@laadvo laadvo requested a review from a team as a code owner September 19, 2023 13:02
@laadvo laadvo added the small label Sep 19, 2023
Copy link
Contributor

@DominikRemo DominikRemo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes LGTM. Test succeeds locally

Copy link
Contributor

@Strohgelaender Strohgelaender left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code lgtm
thanks a lot for figuring this out.

Copy link
Collaborator

@MaximilianAnzinger MaximilianAnzinger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@laadvo laadvo added this to the 6.5.1 milestone Sep 20, 2023
@krusche krusche modified the milestones: 6.5.1, 6.5.2 Sep 21, 2023
Copy link
Contributor

@dearjasmina dearjasmina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code lgtm

@krusche krusche merged commit 08517d3 into develop Sep 22, 2023
16 of 19 checks passed
@krusche krusche deleted the development/fix-PostIntegrationTest branch September 22, 2023 13:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants