Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: sympy log parser fails to recognize the last test case #212

Merged
merged 1 commit into from
Oct 7, 2024

Conversation

yuxqiu
Copy link
Contributor

@yuxqiu yuxqiu commented Aug 26, 2024

What does this implement/fix?

Add additional checking and cleaning steps to the sympy log parser.

Previously, the sympy log parser could not parse the last line of the log file. For example, the following log (where test_intersection_of_duplicate_sets is a test I added manually):

...
test_issue_9447 ok
test_issue_10337 ok
test_issue_10326 ok
test_issue_2799 ok
test_issue_9706 ok
test_issue_8257 ok
test_issue_10931 ok
test_issue_11174 ok
test_finite_set_intersection ok
test_union_intersection_constructor ok
test_Union_contains ok
test_issue_16878b f
test_intersection_of_duplicate_sets F                                     [FAIL]

The current log parser will ignore the last line because there is a text [FAIL] (or [OK]) at the end of that line. This text will appear in the log when all the test functions in the file have been executed. This logic is in leaving_filename function, which will be called at the end of the test_file function.

To fix this, we can simply check to see if there is a [FAIL] or [OK] at the end of the line, and if there is, we can simply delete it and strip the line again.

@yuxqiu yuxqiu changed the title fix: sympy log parser fails to recognize whether the last test case p… fix: sympy log parser fails to recognize the last test case Aug 26, 2024
@@ -203,6 +203,9 @@ def parse_log_sympy(log: str) -> dict[str, str]:
for line in log.split("\n"):
line = line.strip()
if line.startswith("test_"):
if line.endswith("[FAIL]") or line.endswith("[OK]"):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(could've just done line.strip().removesuffix("...").removesuffix("...").strip(), but this should work too)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sense! Thanks for your review.

@john-b-yang
Copy link
Member

Thanks @yuxqiu for the fix and @klieret for the review, merged!

@john-b-yang john-b-yang merged commit 88b9348 into swe-bench:main Oct 7, 2024
1 check passed
@yuxqiu yuxqiu deleted the sympy-log-parser branch October 7, 2024 17:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants