Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Quiz exercises: Fix an issue with the evaluation when practice mode submissions are available #9821

Conversation

KonstiAnon
Copy link
Contributor

@KonstiAnon KonstiAnon commented Nov 19, 2024

Checklist

General

Server

  • Important: I implemented the changes with a very good performance and prevented too many (unnecessary) and too complex database calls.
  • I strictly followed the principle of data economy for all database calls.
  • I strictly followed the server coding and design guidelines.
  • I added pre-authorization annotations according to the guidelines and checked the course groups for all new REST Calls (security).
  • I documented the Java code using JavaDoc style.

Motivation and Context

In rare cases, the scheduled evaluation of a quiz may not automatically occur after the deadline has passed. If the course organizer subsequently opens the quiz for practice without first evaluating it, and users submit answers in practice mode, the evaluation process will no longer function correctly. This happens because the system's logic prioritizes the most recent submissions for each participant, which in this case are the practice submissions. As a result, the unevaluated original submissions are overlooked and skipped during the evaluation process.

Description

I updated the evaluation function to address this issue. For quizzes with a deadline, the function now selects only the submissions made before the deadline. For exam quizzes without a deadline, it retains the existing logic of selecting the submission with the highest ID. With this updated approach, even if submissions are made after the deadline, only the relevant ones are evaluated, ensuring accurate results.

Steps for Testing

Prerequisites:

  • 1 Instructor
  • 1 Student
  • 1 Quiz Exercise
  1. Log in to Artemis (Instructor)
  2. Navigate to Course Administration
  3. Create a new quiz with a due date in the future
  4. Log in as Student
  5. Navigate to Course Page
  6. Participate in the quiz
  7. Before the due date arrives, terminate the Artemis Server
  8. Wait for due date to pass
  9. Start Artemis Server
  10. Log in as Instructor
  11. Naviate to Course Administration
  12. Navigate to Quiz Excercise
  13. Open quiz for practice mode
  14. Log in as Student
  15. Particpipate in quiz practice
  16. Log in as Instructor
  17. Naviate to Course Administraction
  18. Navigate to Quiz excercise
  19. Evaluate Quiz

Testserver States

Note

These badges show the state of the test servers.
Green = Currently available, Red = Currently locked
Click on the badges to get to the test servers.







Review Progress

Performance Review

  • I (as a reviewer) confirm that the client changes (in particular related to REST calls and UI responsiveness) are implemented with a very good performance even for very large courses with more than 2000 students.
  • I (as a reviewer) confirm that the server changes (in particular related to database calls) are implemented with a very good performance even for very large courses with more than 2000 students.

Code Review

  • Code Review 1
  • Code Review 2

Manual Tests

  • Test 1
  • Test 2

Exam Mode Test

  • Test 1
  • Test 2

Summary by CodeRabbit

  • New Features

    • Enhanced evaluation logic for quiz submissions, ensuring only valid submissions before the deadline are considered.
    • Improved handling of existing results to prevent duplicate evaluations.
  • Bug Fixes

    • Refined error handling and logging for scenarios with no valid submissions or multiple submissions.
  • Chores

    • Adjusted the flow of saving submissions, results, and participations to improve database operation reliability.

@github-actions github-actions bot added server Pull requests that update Java code. (Added Automatically!) quiz Pull requests that affect the corresponding module labels Nov 19, 2024
@Hialus Hialus added this to the 7.7.2 milestone Nov 19, 2024
@KonstiAnon KonstiAnon marked this pull request as ready for review November 25, 2024 16:54
@KonstiAnon KonstiAnon requested a review from a team as a code owner November 25, 2024 16:54
Copy link

coderabbitai bot commented Nov 25, 2024

Walkthrough

The changes in the QuizResultService.java file primarily focus on the evaluateSubmissions method. Enhancements include a filtering mechanism to consider only submissions made before the quiz deadline and selecting the highest ID submission when multiple valid submissions exist. The method also refines the handling of existing results by skipping evaluations when rated results are found and creating new results when necessary. Improved error handling and logging have been implemented, along with adjustments to the flow of saving entities to ensure individual saving of submissions, results, and participations.

Changes

File Path Change Summary
src/main/java/de/tum/cit/aet/artemis/quiz/service/QuizResultService.java Modified evaluateSubmissions method to enhance submission evaluation logic, including filtering, handling existing results, improved error handling, and adjusted saving flow.

Possibly related PRs

Suggested labels

tests, bugfix, assessment

Suggested reviewers

  • az108
  • krusche
  • pzdr7
  • JohannesStoehr

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai or @coderabbitai title anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (3)
src/main/java/de/tum/cit/aet/artemis/quiz/service/QuizResultService.java (3)

111-111: Remove unused variable quizDeadline.

The variable quizDeadline is declared but not used in the code. Consider removing it to clean up the code.

Apply this diff to remove the unused variable:

- ZonedDateTime quizDeadline = quizExercise.getDueDate();

158-184: Refactor entity save operations to simplify code and improve maintainability.

Detaching and re-associating the submission with the result leads to multiple save operations and added complexity. Consider refactoring the save operations to leverage cascading persist operations or properly configured entity relationships to handle save order without manual detachment and reattachment.

Here's a possible refactor:

- // Detach submission to maintain proper save order
- result.setSubmission(null);

- // Save entities individually
- submissionRepository.save(quizSubmission);
- result = resultRepository.save(result);

- // Update participation with new result
- participation.addResult(result);
- studentParticipationRepository.save(participation);

- // Re-associate result with submission and save
- result.setSubmission(quizSubmission);
- quizSubmission.addResult(result);
- submissionRepository.save(quizSubmission);

+ // Associate submission with result and add result to participation
+ result.setSubmission(quizSubmission);
+ participation.addResult(result);

+ // Save participation with cascading saves
+ studentParticipationRepository.save(participation);

+ // The result and submission will be saved through cascading
+ // Add result to the set of created results
createdResults.add(result);

By configuring the entity relationships to cascade persist operations appropriately, you can simplify the save operations and reduce the number of explicit saves, improving code readability and maintainability.


Line range hint 92-184: Consider breaking down the evaluateSubmissions method for better readability.

The evaluateSubmissions method is lengthy and contains nested logic, which can make it harder to understand and maintain. Refactoring it into smaller, well-named methods (e.g., filterValidSubmissions, processParticipation, createOrUpdateResult) can improve code clarity and adhere to the single responsibility principle.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 745c7f7 and 9c55a15.

📒 Files selected for processing (1)
  • src/main/java/de/tum/cit/aet/artemis/quiz/service/QuizResultService.java (4 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
src/main/java/de/tum/cit/aet/artemis/quiz/service/QuizResultService.java (1)

Pattern src/main/java/**/*.java: naming:CamelCase; principles:{single_responsibility,small_methods,no_duplication}; db:{perf_queries,datetime_not_timestamp}; rest:{stateless,singleton,delegate_logic,http_only,minimal_dtos}; dtos:{java_records,no_entities,min_data,single_resp}; di:constructor_injection; kiss:simple_code; file_handling:os_indep_paths; practices:{least_access,avoid_transactions,code_reuse,static_member_ref,prefer_primitives}; sql:{param_annotation,uppercase,avoid_subqueries};java:avoid_star_imports

🔇 Additional comments (5)
src/main/java/de/tum/cit/aet/artemis/quiz/service/QuizResultService.java (5)

9-9: Import statement for Optional is appropriate.

The import of java.util.Optional is necessary for the use of Optional instances in the code.


92-93: Javadoc updated to reflect new logic.

The added comments accurately document the new step of filtering out submissions not submitted before the quiz deadline, enhancing code readability and maintainability.


96-97: Clarification in Javadoc about existing rated results.

Including the logic about skipping evaluation if a rated result already exists helps to clearly communicate the method's functionality.


130-142: Filtering submissions based on quiz deadline is implemented correctly.

The code correctly filters out submissions made after the quiz deadline and selects the submission with the highest ID among the valid ones. This ensures that only eligible submissions are evaluated, aligning with the PR objectives.


150-155: Properly skip evaluation when a rated result already exists.

The logic to check for an existing rated result and skip evaluation if present prevents duplicate results and unnecessary processing.

Copy link
Contributor

@iyannsch iyannsch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code LGTM, good job finding this bug!

Copy link
Member

@Hialus Hialus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested locally. Broken live quizzes can be fixed with it and exam quizzes are also still working.
Code also looks good.

Thanks for the fix 🚀

@Hialus Hialus added the maintainer-approved The feature maintainer has approved the PR label Nov 25, 2024
Copy link
Member

@krusche krusche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code looks good to me 👍

@krusche krusche changed the title Quiz exercises: Fix evaluation logic after practice mode submissions were performed Quiz exercises: Fix an issue with the evaluation when practice mode submissions are available Nov 25, 2024
@krusche krusche merged commit bda8f19 into develop Nov 25, 2024
31 of 35 checks passed
@krusche krusche deleted the bugfix/quiz-exercises/fix-evaluation-after-practice-submissions branch November 25, 2024 23:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
maintainer-approved The feature maintainer has approved the PR quiz Pull requests that affect the corresponding module ready for review server Pull requests that update Java code. (Added Automatically!)
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

4 participants