Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Development: Enable parallel server test execution #6796

Merged
merged 105 commits into from
Sep 22, 2023
Merged
Show file tree
Hide file tree
Changes from 103 commits
Commits
Show all changes
105 commits
Select commit Hold shift + click to select a range
e260fcc
Enable parallel Unit Testing
DominikRemo Jun 30, 2023
110af5b
Add systemProperties to build.gradle
DominikRemo Jun 30, 2023
d347901
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Jul 3, 2023
3eb31e8
Try to parallelise Integration Tests
DominikRemo Jul 3, 2023
b5af8c8
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Jul 10, 2023
3dc5368
Added new SpringBootTest
DominikRemo Jul 10, 2023
67aeb55
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Jul 11, 2023
06f7042
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Jul 13, 2023
b01f511
Rebalanced groups for more consistent testing time
DominikRemo Jul 13, 2023
3bc2b87
Clean up code
DominikRemo Jul 14, 2023
ab6d48e
Edit order of tests and allow 5 server starts
DominikRemo Jul 14, 2023
cdbd3c0
Disable failing Tests to get Coverage Report
DominikRemo Jul 18, 2023
21754ca
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Jul 20, 2023
35cabaf
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Jul 24, 2023
040f9e2
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Jul 24, 2023
dd0ee3f
Merge branch 'develop' into development/enable-parallel-unit-testing
laadvo Jul 25, 2023
461baea
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Jul 26, 2023
2a11ade
Testing File Logging - Not yet fully functioning
DominikRemo Jul 26, 2023
8bb7449
Run unit-tests sequentially, since they do not get their own Root Logger
DominikRemo Jul 27, 2023
e0ac1d0
Implemented Discriminator for SiftingAppender
DominikRemo Jul 27, 2023
67d61d8
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Jul 28, 2023
24e2799
Remove @Disabled & @Execution(SAME_THREAD)
DominikRemo Jul 28, 2023
f1d37a1
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Jul 31, 2023
bb4426f
Improve logging - some tasks are still not mapped to the right test
DominikRemo Aug 2, 2023
d380a01
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Aug 7, 2023
dd32fd4
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Aug 8, 2023
1f3f258
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Aug 10, 2023
f0c9334
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Aug 14, 2023
058c578
Added information to logs for debugging
DominikRemo Aug 16, 2023
2d8efc1
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Aug 17, 2023
318326c
New debug information
DominikRemo Aug 18, 2023
07fd40c
fix quiz tests
laadvo Aug 19, 2023
4c6532c
Remove additional console output and appender
DominikRemo Aug 21, 2023
a12b753
Merge branch 'develop' into development/enable-parallel-unit-testing
laadvo Aug 22, 2023
8ad8c88
remove processCachedQuizSubmissions() from test
laadvo Aug 23, 2023
78729dd
try fixing more tests
laadvo Aug 23, 2023
9784c30
improve test
laadvo Aug 23, 2023
7ddf022
Merge branch 'develop' into development/enable-parallel-unit-testing
laadvo Aug 23, 2023
c214939
Implemented appender for parallel console logging
DominikRemo Aug 24, 2023
3b6b5dd
Replaced List<byte[]> with ByteArrayOutputStream
DominikRemo Aug 25, 2023
a9cc6db
Merge branch 'develop' into development/enable-parallel-unit-testing
laadvo Aug 25, 2023
8521f1e
fix testQuizSubmitLiveMode_badRequest_alreadySubmitted after merge
laadvo Aug 25, 2023
14aa19c
Create flaky_test_detection.sh
DominikRemo Aug 25, 2023
348f3da
Add summary for flaky test detection
DominikRemo Aug 28, 2023
154a2e9
Fix LongFeedbackResourceIntegrationTest
DominikRemo Aug 28, 2023
96ef094
fix rateNotActivatedHintForAnExerciseForbidden()
laadvo Aug 29, 2023
d812137
Fix NotificationResourceIntegrationTest
DominikRemo Aug 29, 2023
aef3c4b
Fix submitComplaintResponse_examExercise()
DominikRemo Aug 31, 2023
07f6f54
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Aug 31, 2023
8489d93
Merge branch 'develop' into development/enable-parallel-unit-testing
laadvo Sep 1, 2023
2cf967d
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Sep 2, 2023
ae58c10
Fix CourseServiceTest after merge
DominikRemo Sep 2, 2023
483c1ea
Merge remote-tracking branch 'origin/development/enable-parallel-unit…
DominikRemo Sep 2, 2023
9172165
Added documentation for logging utils
DominikRemo Sep 2, 2023
01ea983
Merge branch 'develop' into development/enable-parallel-unit-testing
laadvo Sep 4, 2023
da59c7a
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Sep 5, 2023
ba088f6
Make AbstractSpringIntegrationTest abstract
DominikRemo Sep 5, 2023
5eaca3f
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Sep 7, 2023
d9554cd
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Sep 8, 2023
4a8a243
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Sep 9, 2023
aea5631
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Sep 13, 2023
5e472d5
fix quiz participation test
laadvo Sep 13, 2023
45c83bc
Merge branch 'develop' into development/enable-parallel-unit-testing
laadvo Sep 13, 2023
650a66b
fix NotificationScheduleServiceTest
laadvo Sep 13, 2023
12064ce
Fix flaky ProgrammingExerciseScheduleServiceTest
DominikRemo Sep 13, 2023
41348c0
fix lockAllRepositories
laadvo Sep 13, 2023
7ee2957
Merge remote-tracking branch 'origin/development/enable-parallel-unit…
laadvo Sep 13, 2023
3ccafe1
Remove new comments from ProgrammingExerciseScheduleServiceTest
DominikRemo Sep 14, 2023
6ff4689
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Sep 14, 2023
4d180b8
Add constants for delay and timeout
DominikRemo Sep 14, 2023
da55330
Merge remote-tracking branch 'origin/development/enable-parallel-unit…
DominikRemo Sep 14, 2023
f5e5835
Isolated QuizCacheTest
DominikRemo Sep 14, 2023
01f9931
Increase delay in NotificationScheduleServiceTest
DominikRemo Sep 14, 2023
b50b752
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Sep 14, 2023
9e3eabd
fix DatabaseQueryCountTest
laadvo Sep 14, 2023
2cc4f99
Increase delay in NotificationScheduleServiceTest
DominikRemo Sep 14, 2023
6824f44
Merge remote-tracking branch 'origin/development/enable-parallel-unit…
DominikRemo Sep 14, 2023
048034f
Change Order for Test Class Execution
DominikRemo Sep 14, 2023
8456743
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Sep 15, 2023
37aeb8d
Merge branch 'develop' into development/enable-parallel-unit-testing
laadvo Sep 18, 2023
f4a0b73
Please work -.-
DominikRemo Sep 18, 2023
38ee34c
Merge remote-tracking branch 'origin/development/enable-parallel-unit…
DominikRemo Sep 18, 2023
1030ef7
Filter Maven Blackbox tests from TemplateIntegrationTest
DominikRemo Sep 18, 2023
877fb5c
please work 2
laadvo Sep 18, 2023
e295134
Merge remote-tracking branch 'origin/development/enable-parallel-unit…
laadvo Sep 18, 2023
42e9954
add await in testGetExerciseStatsForCourseOverview
laadvo Sep 18, 2023
138dde1
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Sep 18, 2023
af32e46
add new quiz tests
laadvo Sep 18, 2023
d4051ca
Test isolation of two QuizSubmissionIntegrationTests
DominikRemo Sep 18, 2023
94b1aef
add await to testGetCourseManagementDetailData
laadvo Sep 19, 2023
9a375fd
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Sep 19, 2023
a95cd18
Fix mockUpdatePlanRepository() after merge
DominikRemo Sep 19, 2023
ed16b37
add await to ParticipantScoreIntegrationTest and ExerciseScoresChartI…
laadvo Sep 19, 2023
acb3057
Isolated getParticipation_quizExerciseStartedAndSubmissionAllowed
DominikRemo Sep 19, 2023
89259bf
add quiz cache tests
laadvo Sep 19, 2023
1b1adc5
Merge remote-tracking branch 'origin/development/enable-parallel-unit…
laadvo Sep 19, 2023
68c1188
fix shouldRecalculateScoreWithTestCaseBonusAndExerciseBonus
laadvo Sep 20, 2023
8fdfc5a
improve IrisMessageIntegrationTest > sendOneMessage
DominikRemo Sep 20, 2023
da44406
Fix fixable Teamscale issues
DominikRemo Sep 20, 2023
e75dc6c
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Sep 21, 2023
6fa23a2
implement feedback from Lucas
laadvo Sep 21, 2023
5af66c1
Merge remote-tracking branch 'origin/development/enable-parallel-unit…
laadvo Sep 21, 2023
b8b389d
Implement feedback from Maximilian
DominikRemo Sep 21, 2023
e042540
Merge branch 'develop' into development/enable-parallel-unit-testing
DominikRemo Sep 22, 2023
97ed064
Fix Server and Client Style after merge
DominikRemo Sep 22, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -164,8 +164,9 @@ private void scheduleNotificationForAssessedExercisesSubmissions(Exercise exerci
* @return true if the time is valid else false
*/
private boolean checkIfTimeIsCorrectForScheduledTask(ZonedDateTime relevantTime) {
// only send a notification if relevantTime is defined and not in the future (i.e. in the range [now-2 minutes, now]) (due to possible delays in scheduling)
return relevantTime != null && !relevantTime.isBefore(ZonedDateTime.now().minusMinutes(2)) && !relevantTime.isAfter(ZonedDateTime.now());
// Only send a notification if relevantTime is defined and close to the current time (i.e. in the range [now-2 minutes, now+2 minutes]) (due to possible delays in
// scheduling)
return relevantTime != null && !relevantTime.isBefore(ZonedDateTime.now().minusMinutes(2)) && !relevantTime.isAfter(ZonedDateTime.now().plusMinutes(2));
}

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@

import javax.validation.Valid;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.messaging.handler.annotation.DestinationVariable;
import org.springframework.messaging.handler.annotation.MessageMapping;
import org.springframework.messaging.handler.annotation.Payload;
Expand All @@ -14,13 +12,12 @@
import de.tum.in.www1.artemis.domain.quiz.QuizSubmission;
import de.tum.in.www1.artemis.exception.QuizSubmissionException;
import de.tum.in.www1.artemis.security.SecurityUtils;
import de.tum.in.www1.artemis.service.*;
import de.tum.in.www1.artemis.service.QuizSubmissionService;
import de.tum.in.www1.artemis.service.WebsocketMessagingService;

@Controller
public class QuizSubmissionWebsocketService {

private static final Logger log = LoggerFactory.getLogger(QuizSubmissionWebsocketService.class);

private final QuizSubmissionService quizSubmissionService;

private final WebsocketMessagingService websocketMessagingService;
Expand All @@ -45,29 +42,11 @@ public void saveSubmission(@DestinationVariable Long exerciseId, @Valid @Payload
// Without this, custom jpa repository methods don't work in websocket channel.
SecurityUtils.setAuthorizationObject();
try {
QuizSubmission updatedQuizSubmission = quizSubmissionService.saveSubmissionForLiveMode(exerciseId, quizSubmission, principal.getName(), false);
// send updated submission over websocket (use a thread to prevent that the outbound channel blocks the inbound channel (e.g. due a slow client))
// to improve the performance, this is currently deactivated: slow clients might lead to bottlenecks so that more important messages can not be distributed any more
// new Thread(() -> sendSubmissionToUser(username, exerciseId, quizSubmission)).start();

// log.info("WS.Inbound: Sent quiz submission (async) back to user {} in quiz {} after {} µs ", principal.getName(), exerciseId, (System.nanoTime() - start) / 1000);
quizSubmissionService.saveSubmissionForLiveMode(exerciseId, quizSubmission, principal.getName(), false);
}
catch (QuizSubmissionException ex) {
// send error message over websocket (use Async to prevent that the outbound channel blocks the inbound channel (e.g. due a slow client))
websocketMessagingService.sendMessageToUser(principal.getName(), "/topic/quizExercise/" + exerciseId + "/submission", new WebsocketError(ex.getMessage()));
}
}

/**
* Should be invoked using a thread asynchronously
*
* @param username the user who saved / submitted the quiz submission
* @param exerciseId the quiz exercise id
* @param quizSubmission the quiz submission that is returned back to the user
*/
private void sendSubmissionToUser(String username, Long exerciseId, QuizSubmission quizSubmission) {
long start = System.nanoTime();
websocketMessagingService.sendMessageToUser(username, "/topic/quizExercise/" + exerciseId + "/submission", quizSubmission);
log.info("WS.Outbound: Sent quiz submission to user {} in quiz {} in {} µs ", username, exerciseId, (System.nanoTime() - start) / 1000);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@

import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.extension.ExtendWith;
import org.junit.jupiter.api.parallel.Execution;
import org.junit.jupiter.api.parallel.ExecutionMode;
import org.junit.jupiter.api.parallel.ResourceLock;
import org.mockito.Mockito;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
Expand Down Expand Up @@ -57,6 +60,8 @@
@SpringBootTest
@AutoConfigureMockMvc
@ExtendWith(SpringExtension.class)
@Execution(ExecutionMode.CONCURRENT)
@ResourceLock("AbstractSpringIntegrationBambooBitbucketJiraTest")
@AutoConfigureEmbeddedDatabase
// NOTE: we use a common set of active profiles to reduce the number of application launches during testing. This significantly saves time and memory!
@ActiveProfiles({ SPRING_PROFILE_TEST, "artemis", "bamboo", "bitbucket", "jira", "ldap", "scheduling", "athena", "apollon", "iris" })
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
package de.tum.in.www1.artemis;

import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.*;
import static tech.jhipster.config.JHipsterConstants.SPRING_PROFILE_TEST;

Expand All @@ -13,6 +11,9 @@
import org.gitlab4j.api.models.PipelineStatus;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.extension.ExtendWith;
import org.junit.jupiter.api.parallel.Execution;
import org.junit.jupiter.api.parallel.ExecutionMode;
import org.junit.jupiter.api.parallel.ResourceLock;
import org.mockito.Mockito;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc;
Expand Down Expand Up @@ -44,6 +45,8 @@
@SpringBootTest
@AutoConfigureMockMvc
@ExtendWith(SpringExtension.class)
@Execution(ExecutionMode.CONCURRENT)
@ResourceLock("AbstractSpringIntegrationGitlabCIGitlabSamlTest")
@AutoConfigureEmbeddedDatabase
// NOTE: we use a common set of active profiles to reduce the number of application launches during testing. This significantly saves time and memory!
@ActiveProfiles({ SPRING_PROFILE_TEST, "artemis", "gitlabci", "gitlab", "saml2", "scheduling" })
Expand Down
Loading
Loading